-- Logs begin at Mon 2022-10-17 09:43:58 UTC, end at Mon 2022-10-17 10:49:07 UTC. -- Oct 17 10:11:04.940675 dv-cinder-860209-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 17 10:11:04.950023 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 17 10:11:04 2022] *** Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: nodename: dv-cinder-860209-5-iscsi Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: machine: x86_64 Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: clock source: unix Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: pcre jit disabled Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: detected number of CPU cores: 4 Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: current working directory: / Oct 17 10:11:04.952034 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: detected binary path: /usr/bin/uwsgi-core Oct 17 10:11:04.952564 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: your processes number limit is 47680 Oct 17 10:11:04.952564 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: your memory page size is 4096 bytes Oct 17 10:11:04.952564 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: detected max file descriptor number: 2048 Oct 17 10:11:04.952564 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: lock engine: pthread robust mutexes Oct 17 10:11:04.952564 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: thunder lock: enabled Oct 17 10:11:04.952564 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 17 10:11:04.952564 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 17 10:11:04.991978 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: Python main interpreter initialized at 0x55acb32c19c0 Oct 17 10:11:04.991978 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: python threads support enabled Oct 17 10:11:04.991978 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: your server socket listen backlog is limited to 100 connections Oct 17 10:11:04.991978 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: your mercy for graceful operations on workers is 90 seconds Oct 17 10:11:04.992322 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: mapped 403077 bytes (393 KB) for 2 cores Oct 17 10:11:04.992415 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: *** Operational MODE: preforking *** Oct 17 10:11:04.992652 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: *** uWSGI is running in multiple interpreter mode *** Oct 17 10:11:04.992652 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: spawned uWSGI master process (pid: 81394) Oct 17 10:11:04.992848 dv-cinder-860209-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 17 10:11:04.993406 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: spawned uWSGI worker 1 (pid: 81406, cores: 1) Oct 17 10:11:04.993736 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: spawned uWSGI worker 2 (pid: 81407, cores: 1) Oct 17 10:11:04.994812 dv-cinder-860209-5-iscsi devstack@keystone.service[81394]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 17 10:11:09.351111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 10:11:09.351111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: from cryptography.utils import int_from_bytes Oct 17 10:11:09.351877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 10:11:09.351877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: from cryptography.utils import int_from_bytes Oct 17 10:11:09.351999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 10:11:09.351999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: from cryptography.utils import int_from_bytes Oct 17 10:11:09.352400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 10:11:09.352400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: from cryptography.utils import int_from_bytes Oct 17 10:11:09.401248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 10:11:09.401434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 10:11:09.401532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 10:11:09.401626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] command line args: [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 10:11:09.401707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 10:11:09.401837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] command line args: [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 10:11:09.401926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 10:11:09.402042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ================================================================================ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 10:11:09.402123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 10:11:09.402278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ================================================================================ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 10:11:09.402387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] admin_token = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.402566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] admin_token = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.402660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_dir = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.402740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.402818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_dir = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.402922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_source = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] debug = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403173 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_source = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] debug = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.403981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404504 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_config_append = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_config_append = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.404943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_dir = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_dir = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405449 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.405958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.406962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407576 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.407944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.408025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.408196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.408284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.408443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.408522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.408722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.408970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.409153 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] publish_errors = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.409240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.409379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] publish_errors = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.409465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.409608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.409692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.409773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.409846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] standard_threads = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] standard_threads = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.410978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_journal = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_json = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_journal = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_stderr = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_json = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_syslog = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.411947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_stderr = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.412025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.412099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_syslog = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.412788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 10:11:09.412873 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.413033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.413114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.413283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.413362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.413521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.413606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.413770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.413856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.414009 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.414089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.414249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.414352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.414510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.414590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.414757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.414837 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.415960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416448 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.416999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.417177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.417264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.417422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.417579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.417657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.417735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.417892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.417971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.418148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.418236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.418421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.418513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.418671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.418753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.418903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.418994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.419135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.419220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.419365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.419444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.419595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.419682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.419835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.419916 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.420079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.420160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.420309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.420398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.420538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.420618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.420772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.420857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.421952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.422981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.423130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.423284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.423360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.423431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.423584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.423670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.423816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.423903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.424979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.425065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.425207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.425327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.425448 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.425555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.425682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.425793 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.425920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.426941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.427115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.427196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.427335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.427407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.427573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.427668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.427796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.427901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.428639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.428791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.428952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.429092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.429224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.429369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.429501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.429635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.429768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.429902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.430036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.430170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.430294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.430443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.430443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.430698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.430833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.430959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.431082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.431207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.431345 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.431470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.431588 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.431715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.431845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.431972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.432110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.432236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.432364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.432492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.432627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.432753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.432899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.433024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.433150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.433277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.433405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.433530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.433663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.433794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.433927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.434052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.434183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.434308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.434502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.434630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.434756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.434881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.435006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.435131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.435264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.435393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.435521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.435646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.435771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.435902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.436103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.436242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.436369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.436496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.436625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.436764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.436894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.437026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.437180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.437304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.437433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.437560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.437693 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.437827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.437954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.438091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.438219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.438359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.438494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.438634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.438768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.438896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.439021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.439147 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.439274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.439408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.439540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.439676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.439804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.439944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.440097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.external = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.440234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.440362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.440496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.external = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.440625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.440756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.440888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.441016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.441144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.password = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.441281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.441409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.token = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.441535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.password = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.441674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.441808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.token = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.441936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.442071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.442199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.442376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.442505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.442642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.442771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.442910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.443038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.443165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.443298 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.443426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.443590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.443732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.443860 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.443992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.444120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.444248 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.444382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.444519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.444654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.444781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.444909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.445038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.445165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.445310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.445439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.445566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.445693 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.445819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.445946 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.446083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.446219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.446362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.446488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.446615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.446742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.446879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.447010 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.447148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.447276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.447401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.447538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.447667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.447795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.447922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.448048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.448176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.448310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.448439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.448566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.448692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.448819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.448955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.449090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.449220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.449347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.449473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.449600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.449726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.449860 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.449987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.450116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.450246 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.450412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.450540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.450689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.450818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.450944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.451072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.451199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.451329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.451531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.451662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.451793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.451920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.452023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.452194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.452332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.452471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.452599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.452726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.452853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.452980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.453115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.453242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.453372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.453500 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.453627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.453754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.453891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.454027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.454155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.454294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.454442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.454575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.454713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.454842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.454971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.455100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.455228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.455358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.455496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.455627 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.455757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.455885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.456024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.456153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.456290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.456417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.456547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.456677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.456806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.456935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.457075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.457206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.457335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.457461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.457587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.457721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.457857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.457982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.458107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.458232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.458416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.458552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.458681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.458809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.458936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.459062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.459187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.459324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.459453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.459580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.459717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.459848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.459974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.460109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.460237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.460364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.460490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.460617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.460745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.460880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.461007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.461134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.461261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.461399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.461526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.461661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.461793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.461922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.462048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.462175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.462302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.462470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.462602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.462732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.462870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.463000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.463129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.463275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.463406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.463406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.463649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.463778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.463907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.464936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.465182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.465308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.465433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.465572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.465702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.465832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.465963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.466094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.466223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.466405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.466533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.466663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.466799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.466928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.467056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.467191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.467320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.467447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.467573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.467702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.467836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.467974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.468102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.468232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.468360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.468488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.468624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.468759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.468887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.469014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.469148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.469275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.469406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.469543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.469671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.469798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.469924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.470050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.470179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.470344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.470472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.470602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.470730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.470859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.470987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.471126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.471513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.471644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.471780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.471903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.472032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.472169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.472299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.472437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.472566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.472696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.472825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.472962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.473091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.473238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.473360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.473489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.473618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.473755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.473890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.474016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.474140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.474279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.474448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.474587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.474715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.475330 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.475460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.475593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.475729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.475858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.475985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.476112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.476239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.476367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.476502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.476639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.476767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.476893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.477021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.477147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.477147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.477147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.477566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.477701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.477828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.477955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.478089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.478217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.478367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.478495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.478620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.478745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.478881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.479007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.479132 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.479257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.479382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.479509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.479642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.479768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.479893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.480018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.480152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.480279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.480412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.480538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.480665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.480789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.480914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.481038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.481171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.481301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.481426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.481552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.481678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.481818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.481952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.482078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.482204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.482347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.482473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.482615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.482745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.482872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.482999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.483125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.483254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.483389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.483527 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.483655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.483783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.483910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.484932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.485180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.485309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.485436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.485562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.485697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.485697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.485944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.486071 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.486155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.486374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.486511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.486642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.486772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.486902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.487040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.487169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.487310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.487439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.487566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.487691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.487817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.487942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.488075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.488201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.488326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.488450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.488575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.488714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.488850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.488976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.489102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.489228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.489353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.489478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.489611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.489737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.489863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.489989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.490124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.490257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.491036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.491221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.491356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.491484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.491570 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.491680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.491809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.491950 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.492080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.492217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.492352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.492480 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.492607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.492735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.492861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.493005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.493135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.493262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.493262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.493262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.493614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.493750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.493892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.494023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.494153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.494294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.494448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.494592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.494734 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.494886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.495020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.495158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.495290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.495426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.495556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.495640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.495752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.495878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 10:11:09.496277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 10:11:09.496277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 10:11:09.518305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.518569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.518714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.518846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.518975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.519103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.527167 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.527395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.556335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.556557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.556824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.557058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.610074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.610371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.658265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.658521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.659306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.659543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.813836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.814115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.814256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.814430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.814566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.814716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.814845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.814971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.815098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.815236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.815371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.815371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.838282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.838541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.838763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.838911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.839051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.839187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.839317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.839463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.839596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.839739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.839872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.840005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.840005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.840005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.851225 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.851376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.863922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.864096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.864240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.864387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.864522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.864655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.864788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.864920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.865068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.865208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.865337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.866022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.866211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.866431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 10:11:09.882973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.883137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.883271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 10:11:09.883402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 10:11:09.883530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.883671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.883800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 10:11:09.883929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 10:11:09.884057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.884185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.884325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.884470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.884606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.884738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.884865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.884992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.894279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.894445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.894580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.894713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.894847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.894989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.895129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.895261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.902642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.902975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.905546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.905833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.905963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.906121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.906257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.906441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.906595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.906727 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.906858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.906992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.907124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.907255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.907392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.907522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.907654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.907786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.907918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.908070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.908204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.908337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.908465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.908592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.931676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.932027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.933130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.933417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.933549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.933766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.938113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.938427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.938561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.938695 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.938828 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.938998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.945194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.945376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.945509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.945665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.951111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.951314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.951449 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.951614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.951745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.951904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.952048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.952182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.952315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.952448 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.952580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.952731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.952820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.952957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.953084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.953213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.969221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.969366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.969495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.969629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.973657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.973809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.977629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.977885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.978018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.978187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.982178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.982455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.985334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.985761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:09.988584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.988885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.989027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.989157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.989286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.989443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.989574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.989736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.989875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.990033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.990165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:09.990415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.005902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.006267 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.006512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.006631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.006812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.006895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.007083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.007349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.013659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.014238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.015230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.015799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.016763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.017047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.017332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.017615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.019726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.020014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.020281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.020363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.020564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.020645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.020846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.020967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.021127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.021523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.032679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.032966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.033249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.033532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.033614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.033868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.033949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.034148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.034240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.034458 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.034544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.034836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.035121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.035418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.049984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.051127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.053616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.054634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.056687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.057528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.058190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.059014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.059736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.060020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.060423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.060508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.060749 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.061157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.066844 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.067573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.068310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.069157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.071367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.071767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.072064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.072214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.072355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.072546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.072732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.073015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.080222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.080516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.080800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.081165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.081457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.081740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.088292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.088578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.088866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.089153 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.089431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.089514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 10:11:10.089705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.089799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.089982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.090068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.090262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.090367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.090563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.090658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.090840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.090926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.091117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.091202 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.091393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.091475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.091660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.091791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.091931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.092016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.092203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.092523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.092792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.093064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 10:11:10.127043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55acb32c19c0 pid: 81407 (default app) Oct 17 10:11:10.128506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55acb32c19c0 pid: 81406 (default app) Oct 17 10:11:10.130260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522417a4-baf2-46e6-9d05-77e10c5e7e98 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:10.130573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522417a4-baf2-46e6-9d05-77e10c5e7e98 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:10.130849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522417a4-baf2-46e6-9d05-77e10c5e7e98 None None] PATH_INFO: `/v3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:10.135755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Mon Oct 17 10:11:10 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:16.886161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072da4-376f-4095-aa24-01a6cd834fa0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:16.886788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072da4-376f-4095-aa24-01a6cd834fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:16.886788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072da4-376f-4095-aa24-01a6cd834fa0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:16.891063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:11:16.897175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:16.897753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:16.897753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:16.967889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: INFO dbcounter [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] Registered counter for database keystone Oct 17 10:11:16.986827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG oslo_db.sqlalchemy.engines [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81407) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 10:11:16.993001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writer thread running {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 10:11:17.171930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG passlib.handlers.bcrypt [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81407) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 10:11:17.172236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG passlib.handlers.bcrypt [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81407) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 10:11:17.230308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:17.231944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:17.357194 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 10:11:17.655001 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4072da4-376f-4095-aa24-01a6cd834fa0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:17.725475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: INFO dbcounter [None req-c4072da4-376f-4095-aa24-01a6cd834fa0 None None] Registered counter for database keystone Oct 17 10:11:17.746819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG oslo_db.sqlalchemy.engines [None req-c4072da4-376f-4095-aa24-01a6cd834fa0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 10:11:17.752959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writer thread running {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 10:11:17.869986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4072da4-376f-4095-aa24-01a6cd834fa0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:17.874170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d561b7-555c-477d-a2a6-dc3c7c5b72c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:17.874810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d561b7-555c-477d-a2a6-dc3c7c5b72c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:17.875086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d561b7-555c-477d-a2a6-dc3c7c5b72c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:17.875355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d561b7-555c-477d-a2a6-dc3c7c5b72c7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:17.883493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d561b7-555c-477d-a2a6-dc3c7c5b72c7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:17.935148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING keystone.common.rbac_enforcer.enforcer [None req-41d561b7-555c-477d-a2a6-dc3c7c5b72c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 10:11:17.935686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d561b7-555c-477d-a2a6-dc3c7c5b72c7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:17.944694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Mon Oct 17 10:11:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:11:17.950603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:17.967350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19a90-a826-4d29-ab66-8b8cb0692ad3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:17.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e45fd-d466-4217-9d6a-e661b040b821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:17.973074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e45fd-d466-4217-9d6a-e661b040b821 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:17.973351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e45fd-d466-4217-9d6a-e661b040b821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:17.973619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e45fd-d466-4217-9d6a-e661b040b821 admin admin] PATH_INFO: `/v3/projects/e6150f4298a94f959838ad2351eaefb9` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:17.975748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e45fd-d466-4217-9d6a-e661b040b821 admin admin] RBAC: Authorizing `identity:get_project(project_id=e6150f4298a94f959838ad2351eaefb9)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:18.028535 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f96e45fd-d466-4217-9d6a-e661b040b821 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 10:11:18.028883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e45fd-d466-4217-9d6a-e661b040b821 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:18.033383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Mon Oct 17 10:11:17 2022] GET /identity/v3/projects/e6150f4298a94f959838ad2351eaefb9 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:20.141261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026cd80-5b02-4c45-98e7-ed463ecbfacf None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:20.141726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026cd80-5b02-4c45-98e7-ed463ecbfacf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:20.141823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026cd80-5b02-4c45-98e7-ed463ecbfacf None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:20.145900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:20.151986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f305f94-d600-44b6-9cc7-e227e7e8602e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:20.152403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f305f94-d600-44b6-9cc7-e227e7e8602e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:20.152535 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f305f94-d600-44b6-9cc7-e227e7e8602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:20.172036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8f305f94-d600-44b6-9cc7-e227e7e8602e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:20.173541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8f305f94-d600-44b6-9cc7-e227e7e8602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:20.181577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 10:11:20.450424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026cd80-5b02-4c45-98e7-ed463ecbfacf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:20.462836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026cd80-5b02-4c45-98e7-ed463ecbfacf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:20.466881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff7ec0-77c5-4aac-985a-b539b487bfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:20.467470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7ec0-77c5-4aac-985a-b539b487bfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:20.467744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7ec0-77c5-4aac-985a-b539b487bfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:20.468012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7ec0-77c5-4aac-985a-b539b487bfe4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:20.475875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7ec0-77c5-4aac-985a-b539b487bfe4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:20.477631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7ec0-77c5-4aac-985a-b539b487bfe4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:20.486102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Mon Oct 17 10:11:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:11:20.492269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f305f94-d600-44b6-9cc7-e227e7e8602e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:20.501118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f305f94-d600-44b6-9cc7-e227e7e8602e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:20.505301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f719235-5329-4a98-b069-b139193ffe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:20.507031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f719235-5329-4a98-b069-b139193ffe3a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:20.507314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f719235-5329-4a98-b069-b139193ffe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:20.507583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f719235-5329-4a98-b069-b139193ffe3a admin admin] PATH_INFO: `/v3/users/e9716bb37d07416391c7fe0f922727e4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:20.508902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f719235-5329-4a98-b069-b139193ffe3a admin admin] RBAC: Authorizing `identity:get_user(user_id=e9716bb37d07416391c7fe0f922727e4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:20.510732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f719235-5329-4a98-b069-b139193ffe3a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:20.512231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Mon Oct 17 10:11:20 2022] GET /identity/v3/users/e9716bb37d07416391c7fe0f922727e4 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:22.679998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7ea7d-a943-4a80-9106-0d505c6cea5d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:22.680528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7ea7d-a943-4a80-9106-0d505c6cea5d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:22.680528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7ea7d-a943-4a80-9106-0d505c6cea5d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:22.681891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:22.687202 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59043c2-ee0b-4911-ae49-4215c8518b6b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:22.687617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59043c2-ee0b-4911-ae49-4215c8518b6b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:22.687893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59043c2-ee0b-4911-ae49-4215c8518b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:22.706424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c59043c2-ee0b-4911-ae49-4215c8518b6b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:22.707598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c59043c2-ee0b-4911-ae49-4215c8518b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:22.752970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad91fde-bd49-4a5e-81f8-89e77c59d14d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:22.753251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad91fde-bd49-4a5e-81f8-89e77c59d14d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:22.753530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad91fde-bd49-4a5e-81f8-89e77c59d14d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:22.754898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:11:22.760975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:22.761255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:22.761528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:22.788671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:22.804327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG passlib.handlers.bcrypt [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81406) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 10:11:22.804719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG passlib.handlers.bcrypt [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81406) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 10:11:22.853652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:22.855223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:22.952250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:23.057856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59043c2-ee0b-4911-ae49-4215c8518b6b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:23.083871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:23.164102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59043c2-ee0b-4911-ae49-4215c8518b6b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:23.197603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96f138-ca0f-4cb4-9911-8ebfe85e74ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:23.198288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96f138-ca0f-4cb4-9911-8ebfe85e74ae None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:23.198594 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96f138-ca0f-4cb4-9911-8ebfe85e74ae None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:23.198864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96f138-ca0f-4cb4-9911-8ebfe85e74ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:23.212427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96f138-ca0f-4cb4-9911-8ebfe85e74ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5b96f138-ca0f-4cb4-9911-8ebfe85e74ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:23.221144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:23.222424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:23.223423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:23.224348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:23.224440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:11:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:23.230226 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96f138-ca0f-4cb4-9911-8ebfe85e74ae None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:23.312856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:23.323459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58183018-d11b-45fd-a7d4-34a54e68675f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:23.325688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:23.331813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96f138-ca0f-4cb4-9911-8ebfe85e74ae None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:23.361068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:23.361921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:23.362198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:23.362498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:23.363468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:23.364879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:23.366949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:23.367542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:23.367813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:23.368079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:23.368734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:23.369030 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:23.370424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:23.376061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 10:11:23.376061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 10:11:23.376061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [parameters: {'id': 'e95bddfda36d434a877da3e526220ec5', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 10:11:23.376061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 10:11:23.383223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:23.388728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:11:23.394722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 10:11:23.395671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 10:11:23.397278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 10:11:23.398185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 10:11:23.399386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:11:23.400476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'id': 'e95bddfda36d434a877da3e526220ec5', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 10:11:23.401576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:23.402539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 8/16] 10.222.0.16 () {60 vars in 1137 bytes} [Mon Oct 17 10:11:23 2022] POST /identity/v3/domains => generated 190 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:11:23.403738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:23.491593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370e57e-c48f-4c3d-a057-39f48db4d781 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:23.502181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a2c43-8b17-4f7a-970a-e45bc26e476c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:23.524509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89637a3e-ac89-45b2-98ae-316450e8ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:23.525136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637a3e-ac89-45b2-98ae-316450e8ffab None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:23.525413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637a3e-ac89-45b2-98ae-316450e8ffab None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:23.525691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637a3e-ac89-45b2-98ae-316450e8ffab None admin] PATH_INFO: `/v3/users/e9716bb37d07416391c7fe0f922727e4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:23.526859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89637a3e-ac89-45b2-98ae-316450e8ffab None admin] RBAC: Authorizing `identity:get_user(user_id=e9716bb37d07416391c7fe0f922727e4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:23.528305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89637a3e-ac89-45b2-98ae-316450e8ffab None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:23.529855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:23 2022] GET /identity/v3/users/e9716bb37d07416391c7fe0f922727e4 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:23.534915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:23.535207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89637a3e-ac89-45b2-98ae-316450e8ffab None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:23.535756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:23.536030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:23.536296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:23.551063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:23.557272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:23.558781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:11:23.577638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:23.578677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:11:23.579661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:11:23.579661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:11:23.579661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:23.579661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:11:23.585467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:23.607545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:23.637315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89637a3e-ac89-45b2-98ae-316450e8ffab None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:23.670852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71509663-a9a8-4162-b88d-96869524d4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:23.671548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71509663-a9a8-4162-b88d-96869524d4eb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:23.671826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71509663-a9a8-4162-b88d-96869524d4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:23.672093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71509663-a9a8-4162-b88d-96869524d4eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:23.673190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71509663-a9a8-4162-b88d-96869524d4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:23.674655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71509663-a9a8-4162-b88d-96869524d4eb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:23.676189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:23.682276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71509663-a9a8-4162-b88d-96869524d4eb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:23.685038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa3002-11b3-49a6-bd4b-108bd3964352 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:23.722919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e53f97-7535-4642-856b-ea85355780fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:23.723526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53f97-7535-4642-856b-ea85355780fb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:23.723801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53f97-7535-4642-856b-ea85355780fb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:23.724070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53f97-7535-4642-856b-ea85355780fb None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:23.724774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e53f97-7535-4642-856b-ea85355780fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:23.725078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e53f97-7535-4642-856b-ea85355780fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:23.726472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e53f97-7535-4642-856b-ea85355780fb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:23.743968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:23.766532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:23.788206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71509663-a9a8-4162-b88d-96869524d4eb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:23.823916 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aed9da-58ec-4a36-92e4-9d676b35e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:23.824587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed9da-58ec-4a36-92e4-9d676b35e543 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:23.824862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed9da-58ec-4a36-92e4-9d676b35e543 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:23.825135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed9da-58ec-4a36-92e4-9d676b35e543 None admin] PATH_INFO: `/v3/domains/default/users/e9716bb37d07416391c7fe0f922727e4/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:23.827131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aed9da-58ec-4a36-92e4-9d676b35e543 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e9716bb37d07416391c7fe0f922727e4, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:23.828609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aed9da-58ec-4a36-92e4-9d676b35e543 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:23.947052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 11/21] 10.222.0.16 () {58 vars in 1361 bytes} [Mon Oct 17 10:11:23 2022] PUT /identity/v3/domains/default/users/e9716bb37d07416391c7fe0f922727e4/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:11:26.283971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655de5-8fa9-4396-893c-c53b08338ef8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:26.284475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655de5-8fa9-4396-893c-c53b08338ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:26.284547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655de5-8fa9-4396-893c-c53b08338ef8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:26.285837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:26.304600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6975758-dc83-4945-b4eb-74cb1bf7bd0e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:26.305101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6975758-dc83-4945-b4eb-74cb1bf7bd0e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:26.305197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6975758-dc83-4945-b4eb-74cb1bf7bd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:26.335636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f6975758-dc83-4945-b4eb-74cb1bf7bd0e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:26.337097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f6975758-dc83-4945-b4eb-74cb1bf7bd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:26.353958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baeb02-1e56-4c1e-82da-6a79a34d1629 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:26.354247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baeb02-1e56-4c1e-82da-6a79a34d1629 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:26.354534 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baeb02-1e56-4c1e-82da-6a79a34d1629 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:26.356139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 12/23] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:11:26.361877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e6826-fd3e-4fde-96a6-368c638dc720 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:26.362160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e6826-fd3e-4fde-96a6-368c638dc720 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:26.362456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e6826-fd3e-4fde-96a6-368c638dc720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:26.395394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bb0e6826-fd3e-4fde-96a6-368c638dc720 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:26.397154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bb0e6826-fd3e-4fde-96a6-368c638dc720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:26.446939 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 12/24] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:26.538950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:26.556616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df278a5-461e-44db-a689-203b3c454efe None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:26.556907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df278a5-461e-44db-a689-203b3c454efe None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:26.557181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df278a5-461e-44db-a689-203b3c454efe None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:26.561571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 13/26] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:26.566990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1744-ec52-4611-91c0-1823e214495d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:26.567274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1744-ec52-4611-91c0-1823e214495d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:26.567551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1744-ec52-4611-91c0-1823e214495d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:26.595464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1bfb1744-ec52-4611-91c0-1823e214495d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:26.596837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1bfb1744-ec52-4611-91c0-1823e214495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:26.599981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:26.687814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 14/27] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:26.780790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df278a5-461e-44db-a689-203b3c454efe None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:26.875490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:26.924761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df278a5-461e-44db-a689-203b3c454efe None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:26.972518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:26.973192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:26.973474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:26.973746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:26.986703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:26.987401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb1744-ec52-4611-91c0-1823e214495d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:26.988482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:11:27.009831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:27.010791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:11:27.011613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:11:27.011613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:11:27.011613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:27.011613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 14/28] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:27.018405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:27.183948 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd658a-334b-4ca4-afcc-d38e194919c0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:27.186029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb1744-ec52-4611-91c0-1823e214495d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:27.215604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:27.228477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2b3f6-bb4d-4131-8636-84b9962ac1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:27.229365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2b3f6-bb4d-4131-8636-84b9962ac1fc None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:27.229856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2b3f6-bb4d-4131-8636-84b9962ac1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:27.230342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2b3f6-bb4d-4131-8636-84b9962ac1fc None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:27.231233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2b3f6-bb4d-4131-8636-84b9962ac1fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:27.232144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee9daa-9e9f-4bb8-a573-8e69d64f0e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:27.232761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee9daa-9e9f-4bb8-a573-8e69d64f0e4b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:27.233043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee9daa-9e9f-4bb8-a573-8e69d64f0e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:27.233316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee9daa-9e9f-4bb8-a573-8e69d64f0e4b None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:27.233705 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2b3f6-bb4d-4131-8636-84b9962ac1fc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:27.233958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee9daa-9e9f-4bb8-a573-8e69d64f0e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:27.234263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee9daa-9e9f-4bb8-a573-8e69d64f0e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:27.235687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee9daa-9e9f-4bb8-a573-8e69d64f0e4b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:27.250287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:27.258791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 15/29] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:27.263193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51366ccd-c1a6-45a0-948b-f7ea9eaea00e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:27.266553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51366ccd-c1a6-45a0-948b-f7ea9eaea00e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:27.267034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51366ccd-c1a6-45a0-948b-f7ea9eaea00e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:27.268725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 16/30] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:27.274387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51366ccd-c1a6-45a0-948b-f7ea9eaea00e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:27.389351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 15/31] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Oct 17 10:11:26 2022] POST /identity/v3/roles => generated 219 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:27.389905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51366ccd-c1a6-45a0-948b-f7ea9eaea00e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:27.390481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2b3f6-bb4d-4131-8636-84b9962ac1fc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:27.481245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f4ae3-ed8e-4406-b801-5dad0a9c72d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:27.482259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4ae3-ed8e-4406-b801-5dad0a9c72d1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:27.482778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4ae3-ed8e-4406-b801-5dad0a9c72d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:27.483326 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4ae3-ed8e-4406-b801-5dad0a9c72d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:27.515014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f4ae3-ed8e-4406-b801-5dad0a9c72d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-898f4ae3-ed8e-4406-b801-5dad0a9c72d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:27.524113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:27.525014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:27.525784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 17/32] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:11:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:27.529250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1536a-694b-48e8-8e47-fa41bc818fb2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:27.529250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1536a-694b-48e8-8e47-fa41bc818fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:27.529250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1536a-694b-48e8-8e47-fa41bc818fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:27.541016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2b3f6-bb4d-4131-8636-84b9962ac1fc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:27.563376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cac1536a-694b-48e8-8e47-fa41bc818fb2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:27.564572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cac1536a-694b-48e8-8e47-fa41bc818fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:27.582061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37997bd-4db9-46a2-b19a-891201a7634e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:27.582716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37997bd-4db9-46a2-b19a-891201a7634e None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:27.582997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37997bd-4db9-46a2-b19a-891201a7634e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:27.583275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37997bd-4db9-46a2-b19a-891201a7634e None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:27.584280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37997bd-4db9-46a2-b19a-891201a7634e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:27.585701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37997bd-4db9-46a2-b19a-891201a7634e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:27.626726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 16/33] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:11:27 2022] POST /identity/v3/projects => generated 301 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:27.627926 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37997bd-4db9-46a2-b19a-891201a7634e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:27.650414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:27.696697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 18/34] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:27.704617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:27.760993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37997bd-4db9-46a2-b19a-891201a7634e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:27.799134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:27.799813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:27.800102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:27.800377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:27.801060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:27.801366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:27.803414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:27.817103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 17/35] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:27.823429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1536a-694b-48e8-8e47-fa41bc818fb2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:28.016638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:28.064278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1536a-694b-48e8-8e47-fa41bc818fb2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:28.102976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0dcdd-d2b8-4a31-a111-2a90f1b0f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:28.103616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0dcdd-d2b8-4a31-a111-2a90f1b0f963 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.103896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0dcdd-d2b8-4a31-a111-2a90f1b0f963 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.104167 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0dcdd-d2b8-4a31-a111-2a90f1b0f963 None admin] PATH_INFO: `/v3/users/e9716bb37d07416391c7fe0f922727e4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.105512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0dcdd-d2b8-4a31-a111-2a90f1b0f963 None admin] RBAC: Authorizing `identity:get_user(user_id=e9716bb37d07416391c7fe0f922727e4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:28.107014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0dcdd-d2b8-4a31-a111-2a90f1b0f963 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:28.113692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 19/36] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:27 2022] GET /identity/v3/users/e9716bb37d07416391c7fe0f922727e4 => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:28.114587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0dcdd-d2b8-4a31-a111-2a90f1b0f963 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:28.152473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a233-0596-4ceb-a0b2-78680f6a2ded None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:28.166722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:28.170851 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:28.198589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c919578-d3a1-4680-8795-0026ed957029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:28.199226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c919578-d3a1-4680-8795-0026ed957029 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.199503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c919578-d3a1-4680-8795-0026ed957029 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.199776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c919578-d3a1-4680-8795-0026ed957029 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.200448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c919578-d3a1-4680-8795-0026ed957029 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:28.201832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c919578-d3a1-4680-8795-0026ed957029 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:28.218909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Oct 17 10:11:28 2022] POST /identity/v3/roles => generated 225 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:11:28.220216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387953b-0752-4a74-964c-a208afcd85af None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.220500 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387953b-0752-4a74-964c-a208afcd85af None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.220777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387953b-0752-4a74-964c-a208afcd85af None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.222534 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 19/38] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:28.228338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906fa6-938a-4dd8-9766-02a3d0b1ab92 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.228625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906fa6-938a-4dd8-9766-02a3d0b1ab92 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.228899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906fa6-938a-4dd8-9766-02a3d0b1ab92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.252983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a0906fa6-938a-4dd8-9766-02a3d0b1ab92 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:28.254514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0dcdd-d2b8-4a31-a111-2a90f1b0f963 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:28.255515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a0906fa6-938a-4dd8-9766-02a3d0b1ab92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:28.300978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee249d-aff1-47e7-84cc-03034bb7ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:28.301552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee249d-aff1-47e7-84cc-03034bb7ed17 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.301834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee249d-aff1-47e7-84cc-03034bb7ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.302108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee249d-aff1-47e7-84cc-03034bb7ed17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.303812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee249d-aff1-47e7-84cc-03034bb7ed17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:28.306643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee249d-aff1-47e7-84cc-03034bb7ed17 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:28.308501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 20/39] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:28.315595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee249d-aff1-47e7-84cc-03034bb7ed17 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:28.365317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 20/40] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:28.382935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d078a3-76ee-41de-a076-cc50415bcb83 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.383224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d078a3-76ee-41de-a076-cc50415bcb83 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.383507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d078a3-76ee-41de-a076-cc50415bcb83 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.384923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 21/41] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:11:28.390596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33450c-c56c-4e00-8737-25f0c70ec0b7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.391202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33450c-c56c-4e00-8737-25f0c70ec0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.391202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33450c-c56c-4e00-8737-25f0c70ec0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.410360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9e33450c-c56c-4e00-8737-25f0c70ec0b7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:28.411537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9e33450c-c56c-4e00-8737-25f0c70ec0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:28.454797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee249d-aff1-47e7-84cc-03034bb7ed17 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:28.466610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:28.502746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:28.503806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.503922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 22/42] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:28.504399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.504903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.506213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.domain.id=default, user.id=e9716bb37d07416391c7fe0f922727e4) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:28.506864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:28.508029 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:28.516423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 21/43] 10.222.0.16 () {56 vars in 1322 bytes} [Mon Oct 17 10:11:28 2022] GET /identity/v3/role_assignments?user.id=e9716bb37d07416391c7fe0f922727e4&scope.domain.id=default&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 530 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:28.652136 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33450c-c56c-4e00-8737-25f0c70ec0b7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:28.687710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:28.758954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33450c-c56c-4e00-8737-25f0c70ec0b7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:28.779461 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:28.792522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f93612-000c-4ab2-94cd-b78f3a5b3fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:28.793154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f93612-000c-4ab2-94cd-b78f3a5b3fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.793434 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f93612-000c-4ab2-94cd-b78f3a5b3fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.793706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f93612-000c-4ab2-94cd-b78f3a5b3fc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.794849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f93612-000c-4ab2-94cd-b78f3a5b3fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:28.796296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f93612-000c-4ab2-94cd-b78f3a5b3fc9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:28.797860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 23/44] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:28.803281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f93612-000c-4ab2-94cd-b78f3a5b3fc9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:28.880322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487266d-6791-4df7-a746-f433af49d455 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:28.903722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f93612-000c-4ab2-94cd-b78f3a5b3fc9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:28.905152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:28.915609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc241f5f-4823-4f54-90de-6aeb7056aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:28.916229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241f5f-4823-4f54-90de-6aeb7056aa81 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.916508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241f5f-4823-4f54-90de-6aeb7056aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.916781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241f5f-4823-4f54-90de-6aeb7056aa81 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.917451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc241f5f-4823-4f54-90de-6aeb7056aa81 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:28.918958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc241f5f-4823-4f54-90de-6aeb7056aa81 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:28.934856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 22/45] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Oct 17 10:11:28 2022] POST /identity/v3/roles => generated 223 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:11:28.940620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f21c6-a970-4e57-a521-3bfd2ea9b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:28.941250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f21c6-a970-4e57-a521-3bfd2ea9b98d None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:28.941533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f21c6-a970-4e57-a521-3bfd2ea9b98d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:28.941806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f21c6-a970-4e57-a521-3bfd2ea9b98d None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:28.942521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f21c6-a970-4e57-a521-3bfd2ea9b98d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:28.944644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f21c6-a970-4e57-a521-3bfd2ea9b98d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:28.973907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 24/46] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:11:28 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:11:31.124325 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d1e89-e65f-4eb8-b1e7-69f4846d5542 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:31.124797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d1e89-e65f-4eb8-b1e7-69f4846d5542 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:31.124883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d1e89-e65f-4eb8-b1e7-69f4846d5542 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:31.126235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 23/47] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:31.131963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e24e9-552d-4f1f-935f-a2f71580c806 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:31.132377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e24e9-552d-4f1f-935f-a2f71580c806 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:31.132518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e24e9-552d-4f1f-935f-a2f71580c806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:31.150895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2d0e24e9-552d-4f1f-935f-a2f71580c806 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:31.152066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2d0e24e9-552d-4f1f-935f-a2f71580c806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:31.217018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 25/48] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:31.483255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d1e89-e65f-4eb8-b1e7-69f4846d5542 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:31.514297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:31.585149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d1e89-e65f-4eb8-b1e7-69f4846d5542 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:31.618505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459228c-ea56-4b53-a925-66d78f9e9210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:31.619121 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459228c-ea56-4b53-a925-66d78f9e9210 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:31.619402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459228c-ea56-4b53-a925-66d78f9e9210 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:31.619678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459228c-ea56-4b53-a925-66d78f9e9210 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:31.620782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459228c-ea56-4b53-a925-66d78f9e9210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:31.622262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459228c-ea56-4b53-a925-66d78f9e9210 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:31.623852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 24/49] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:31.629393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e24e9-552d-4f1f-935f-a2f71580c806 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:31.710087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:31.727119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e24e9-552d-4f1f-935f-a2f71580c806 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:31.759051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fa5e1-5c0a-4e5e-a080-b7b97d92bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:31.759646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fa5e1-5c0a-4e5e-a080-b7b97d92bcde None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:31.759920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fa5e1-5c0a-4e5e-a080-b7b97d92bcde None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:31.760184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fa5e1-5c0a-4e5e-a080-b7b97d92bcde None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:31.760829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fa5e1-5c0a-4e5e-a080-b7b97d92bcde None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:31.762272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fa5e1-5c0a-4e5e-a080-b7b97d92bcde None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:31.788513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 26/50] 10.222.0.16 () {60 vars in 1140 bytes} [Mon Oct 17 10:11:31 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:33.921113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9738c8-0dfe-4474-acc4-fae8d537c701 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:33.921646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9738c8-0dfe-4474-acc4-fae8d537c701 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:33.921646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9738c8-0dfe-4474-acc4-fae8d537c701 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:33.923038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 25/51] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:33.928710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f04a-fd96-44db-847f-df572e32fd41 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:33.929094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f04a-fd96-44db-847f-df572e32fd41 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:33.929276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f04a-fd96-44db-847f-df572e32fd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:33.947764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2072f04a-fd96-44db-847f-df572e32fd41 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:33.948930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2072f04a-fd96-44db-847f-df572e32fd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:34.013407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 27/52] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:34.287029 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9738c8-0dfe-4474-acc4-fae8d537c701 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:34.385613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:34.389725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9738c8-0dfe-4474-acc4-fae8d537c701 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:34.423190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7244152-a5c3-49af-9e2e-d98322a1d921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:34.423808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7244152-a5c3-49af-9e2e-d98322a1d921 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:34.424088 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7244152-a5c3-49af-9e2e-d98322a1d921 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:34.424362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7244152-a5c3-49af-9e2e-d98322a1d921 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:34.425458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7244152-a5c3-49af-9e2e-d98322a1d921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:34.426935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7244152-a5c3-49af-9e2e-d98322a1d921 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:34.429202 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 26/53] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:34.434793 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072f04a-fd96-44db-847f-df572e32fd41 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:34.497573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:34.531496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072f04a-fd96-44db-847f-df572e32fd41 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:34.565008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9bfc6-1da8-4971-9218-6468b86c7c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:34.565541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bfc6-1da8-4971-9218-6468b86c7c43 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:34.565822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bfc6-1da8-4971-9218-6468b86c7c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:34.566092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bfc6-1da8-4971-9218-6468b86c7c43 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:34.566781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9bfc6-1da8-4971-9218-6468b86c7c43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:34.568241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9bfc6-1da8-4971-9218-6468b86c7c43 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:34.697624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 28/54] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:11:34 2022] POST /identity/v3/users => generated 275 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:37.826480 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0c304-31ec-4357-bebd-d574a045486c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:37.827593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0c304-31ec-4357-bebd-d574a045486c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:37.828074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0c304-31ec-4357-bebd-d574a045486c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:37.829937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 27/55] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:37.835506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c742-2258-401a-b333-657992383eab None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:37.836331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c742-2258-401a-b333-657992383eab None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:37.836841 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c742-2258-401a-b333-657992383eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:37.858295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5a15c742-2258-401a-b333-657992383eab None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:37.859813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5a15c742-2258-401a-b333-657992383eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:37.952323 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 29/56] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:38.183912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940b78-bfa4-4683-84c1-b6dc4d8219ee None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.184587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940b78-bfa4-4683-84c1-b6dc4d8219ee None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.185072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940b78-bfa4-4683-84c1-b6dc4d8219ee None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.186787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 28/57] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:38.188472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38160b2d-2985-4dc3-9b0a-b011309738a5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.188766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38160b2d-2985-4dc3-9b0a-b011309738a5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.189044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38160b2d-2985-4dc3-9b0a-b011309738a5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.190638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:38.191793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376cd6-63e1-4d45-9bb2-d5d29d41cf98 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.192084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376cd6-63e1-4d45-9bb2-d5d29d41cf98 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.192363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376cd6-63e1-4d45-9bb2-d5d29d41cf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.205497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8e072-246d-4be0-a92c-0e19aa892d41 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.205817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8e072-246d-4be0-a92c-0e19aa892d41 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.206102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8e072-246d-4be0-a92c-0e19aa892d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.218136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1b376cd6-63e1-4d45-9bb2-d5d29d41cf98 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:38.219348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1b376cd6-63e1-4d45-9bb2-d5d29d41cf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:38.239975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-87c8e072-246d-4be0-a92c-0e19aa892d41 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:38.241361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-87c8e072-246d-4be0-a92c-0e19aa892d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:38.333419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:38.348478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:38.349952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376cd6-63e1-4d45-9bb2-d5d29d41cf98 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:38.374827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 31/60] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:38.462317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ec12-9e84-4914-82fa-84034d84bdcb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.463007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ec12-9e84-4914-82fa-84034d84bdcb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.463479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ec12-9e84-4914-82fa-84034d84bdcb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.465022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 32/61] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:11:38.474126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850a9a1-bce5-4f0e-91d6-a891d4634451 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.474495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850a9a1-bce5-4f0e-91d6-a891d4634451 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.474780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850a9a1-bce5-4f0e-91d6-a891d4634451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.484722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:38.489010 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376cd6-63e1-4d45-9bb2-d5d29d41cf98 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:38.510850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b850a9a1-bce5-4f0e-91d6-a891d4634451 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:38.512037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b850a9a1-bce5-4f0e-91d6-a891d4634451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:38.541005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d47bd-d35a-480a-b85b-c658125324a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:38.541662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d47bd-d35a-480a-b85b-c658125324a3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.542341 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d47bd-d35a-480a-b85b-c658125324a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.542639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d47bd-d35a-480a-b85b-c658125324a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.565571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d47bd-d35a-480a-b85b-c658125324a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9d5d47bd-d35a-480a-b85b-c658125324a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:38.571062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:38.572020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:38.572880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:38.582858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 30/62] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:38.589494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d47bd-d35a-480a-b85b-c658125324a3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:38.610032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 33/63] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:38.730305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d47bd-d35a-480a-b85b-c658125324a3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:38.749445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850a9a1-bce5-4f0e-91d6-a891d4634451 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:38.781656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:38.782879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.783770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.784364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.785314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:38.786050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:38.788902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:38.800078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 31/64] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:38.800906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:38.836016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:38.869788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850a9a1-bce5-4f0e-91d6-a891d4634451 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:38.884221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:38.906789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299b2b9-19cc-4d69-b627-af3e1ee06ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:38.907426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299b2b9-19cc-4d69-b627-af3e1ee06ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.907704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299b2b9-19cc-4d69-b627-af3e1ee06ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.908245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299b2b9-19cc-4d69-b627-af3e1ee06ac6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.919731 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299b2b9-19cc-4d69-b627-af3e1ee06ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2299b2b9-19cc-4d69-b627-af3e1ee06ac6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:38.926146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:38.927113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 34/65] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:11:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:11:38.928064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299b2b9-19cc-4d69-b627-af3e1ee06ac6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:38.933969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736200-f3f1-4f1b-b54f-6eaac324aed5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:38.968376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e19300-9ab0-461e-a50f-813abac1024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:38.969007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e19300-9ab0-461e-a50f-813abac1024b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.969287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e19300-9ab0-461e-a50f-813abac1024b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.969561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e19300-9ab0-461e-a50f-813abac1024b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.980025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e19300-9ab0-461e-a50f-813abac1024b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-64e19300-9ab0-461e-a50f-813abac1024b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:38.985295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:38.986229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:38.987026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 32/66] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:11:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:38.987681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c1a63-1862-414a-8484-75051d91673b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:38.987790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c1a63-1862-414a-8484-75051d91673b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:38.988064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c1a63-1862-414a-8484-75051d91673b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:38.989313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 33/67] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:38.990736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c1a63-1862-414a-8484-75051d91673b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.031404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299b2b9-19cc-4d69-b627-af3e1ee06ac6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.066174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054abb27-acfc-4fe2-be82-d16848e00080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.066823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054abb27-acfc-4fe2-be82-d16848e00080 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.067102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054abb27-acfc-4fe2-be82-d16848e00080 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.067369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054abb27-acfc-4fe2-be82-d16848e00080 None admin] PATH_INFO: `/v3/users/10a6976357d0445a9fa2891d68878d0a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.072655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:39.086005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054abb27-acfc-4fe2-be82-d16848e00080 None admin] RBAC: Authorizing `identity:get_user(user_id=10a6976357d0445a9fa2891d68878d0a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.087508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054abb27-acfc-4fe2-be82-d16848e00080 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.089164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 35/68] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:38 2022] GET /identity/v3/users/10a6976357d0445a9fa2891d68878d0a => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.093681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054abb27-acfc-4fe2-be82-d16848e00080 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.097766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c1a63-1862-414a-8484-75051d91673b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.119431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:39.132319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f20f8-eb25-4226-8440-0c821ed6ea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.132884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f20f8-eb25-4226-8440-0c821ed6ea88 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.133154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f20f8-eb25-4226-8440-0c821ed6ea88 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.133430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f20f8-eb25-4226-8440-0c821ed6ea88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.143378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f20f8-eb25-4226-8440-0c821ed6ea88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a42f20f8-eb25-4226-8440-0c821ed6ea88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:39.148680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:39.149607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:39.150415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 34/69] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:39.151487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f20f8-eb25-4226-8440-0c821ed6ea88 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.190865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054abb27-acfc-4fe2-be82-d16848e00080 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.223711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69ca10-025e-4331-b385-76acb98905f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.224332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69ca10-025e-4331-b385-76acb98905f1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.225244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69ca10-025e-4331-b385-76acb98905f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.225601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69ca10-025e-4331-b385-76acb98905f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.226430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69ca10-025e-4331-b385-76acb98905f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:39.226742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69ca10-025e-4331-b385-76acb98905f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.228099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69ca10-025e-4331-b385-76acb98905f1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.235833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 36/70] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.237110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a849d-952b-4f77-9203-31320bcbee15 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.237395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a849d-952b-4f77-9203-31320bcbee15 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.237669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a849d-952b-4f77-9203-31320bcbee15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.253014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f20f8-eb25-4226-8440-0c821ed6ea88 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.254718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2c7a849d-952b-4f77-9203-31320bcbee15 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:39.256032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2c7a849d-952b-4f77-9203-31320bcbee15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:39.286430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.287698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.288124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.288402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.289069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:39.289374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.290810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.298778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 35/71] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.299562 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.302058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:39.326312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 37/72] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:39.327181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a849d-952b-4f77-9203-31320bcbee15 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.351134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:39.418853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e49984-05b3-430d-ac75-e3c65508acd2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.438248 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a849d-952b-4f77-9203-31320bcbee15 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.454572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95bf98-19fa-4724-b182-ff93e8750bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.455205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95bf98-19fa-4724-b182-ff93e8750bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.455483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95bf98-19fa-4724-b182-ff93e8750bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.455759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95bf98-19fa-4724-b182-ff93e8750bd8 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.456889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95bf98-19fa-4724-b182-ff93e8750bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.458365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95bf98-19fa-4724-b182-ff93e8750bd8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.460791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 36/73] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.461378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95bf98-19fa-4724-b182-ff93e8750bd8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.480240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.481089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.481534 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.481968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.482842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:39.483341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.484866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.493285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 38/74] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.494262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.539625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:39.573488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95bf98-19fa-4724-b182-ff93e8750bd8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.589478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:39.606376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7faed-a7de-43ec-8891-661b18fc0375 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.612439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca95c-5e78-47c5-b157-62366c2b4190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.613084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca95c-5e78-47c5-b157-62366c2b4190 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.613367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca95c-5e78-47c5-b157-62366c2b4190 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.613640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca95c-5e78-47c5-b157-62366c2b4190 None admin] PATH_INFO: `/v3/users/10a6976357d0445a9fa2891d68878d0a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.614810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ca95c-5e78-47c5-b157-62366c2b4190 None admin] RBAC: Authorizing `identity:get_user(user_id=10a6976357d0445a9fa2891d68878d0a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.616548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ca95c-5e78-47c5-b157-62366c2b4190 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.618132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 37/75] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/users/10a6976357d0445a9fa2891d68878d0a => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.618903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca95c-5e78-47c5-b157-62366c2b4190 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.643194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c441a0b-a2b3-4cbf-af56-b369a6bdaa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.643816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c441a0b-a2b3-4cbf-af56-b369a6bdaa6a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.644089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c441a0b-a2b3-4cbf-af56-b369a6bdaa6a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.644356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c441a0b-a2b3-4cbf-af56-b369a6bdaa6a None admin] PATH_INFO: `/v3/users/e9716bb37d07416391c7fe0f922727e4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.645471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c441a0b-a2b3-4cbf-af56-b369a6bdaa6a None admin] RBAC: Authorizing `identity:get_user(user_id=e9716bb37d07416391c7fe0f922727e4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.647599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c441a0b-a2b3-4cbf-af56-b369a6bdaa6a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.649334 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 39/76] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/users/e9716bb37d07416391c7fe0f922727e4 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.649980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c441a0b-a2b3-4cbf-af56-b369a6bdaa6a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.725907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca95c-5e78-47c5-b157-62366c2b4190 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.748716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c441a0b-a2b3-4cbf-af56-b369a6bdaa6a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.760148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a68f7-b235-4aa7-b6c9-b772aec45458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.760838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a68f7-b235-4aa7-b6c9-b772aec45458 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.761125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a68f7-b235-4aa7-b6c9-b772aec45458 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.761407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a68f7-b235-4aa7-b6c9-b772aec45458 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/users/10a6976357d0445a9fa2891d68878d0a/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.763718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a68f7-b235-4aa7-b6c9-b772aec45458 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8434c7cd8ed4e18a86583801f3af1f7, user_id=10a6976357d0445a9fa2891d68878d0a, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.765158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a68f7-b235-4aa7-b6c9-b772aec45458 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.773658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:39.779598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 38/77] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:39 2022] PUT /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/users/10a6976357d0445a9fa2891d68878d0a/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:11:39.782198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212fd3b-9b9a-4002-b975-b021c543845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.782873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212fd3b-9b9a-4002-b975-b021c543845d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.783153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212fd3b-9b9a-4002-b975-b021c543845d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.783427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212fd3b-9b9a-4002-b975-b021c543845d None admin] PATH_INFO: `/v3/users/10a6976357d0445a9fa2891d68878d0a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.784624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212fd3b-9b9a-4002-b975-b021c543845d None admin] RBAC: Authorizing `identity:get_user(user_id=10a6976357d0445a9fa2891d68878d0a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.786105 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212fd3b-9b9a-4002-b975-b021c543845d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.788076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 40/78] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/users/10a6976357d0445a9fa2891d68878d0a => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.788701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212fd3b-9b9a-4002-b975-b021c543845d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.790230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a68f7-b235-4aa7-b6c9-b772aec45458 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.815756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:39.912768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a68f7-b235-4aa7-b6c9-b772aec45458 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.913356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212fd3b-9b9a-4002-b975-b021c543845d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:39.951727 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461045e-2fd0-439a-b64a-6cc54b56595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.952256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6153b-77ed-4e64-ac6d-a603b195bd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:39.952399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461045e-2fd0-439a-b64a-6cc54b56595c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.952625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461045e-2fd0-439a-b64a-6cc54b56595c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.952898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461045e-2fd0-439a-b64a-6cc54b56595c None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.953474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6153b-77ed-4e64-ac6d-a603b195bd20 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:39.953889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6153b-77ed-4e64-ac6d-a603b195bd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:39.954126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461045e-2fd0-439a-b64a-6cc54b56595c None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.954317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6153b-77ed-4e64-ac6d-a603b195bd20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:39.955651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461045e-2fd0-439a-b64a-6cc54b56595c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.958290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6153b-77ed-4e64-ac6d-a603b195bd20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:39.958434 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 41/79] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.958508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6153b-77ed-4e64-ac6d-a603b195bd20 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:39.958592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461045e-2fd0-439a-b64a-6cc54b56595c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:39.959422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 39/80] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:39.960184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6153b-77ed-4e64-ac6d-a603b195bd20 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:40.032417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:40.056773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:40.070985 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6153b-77ed-4e64-ac6d-a603b195bd20 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:40.077052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461045e-2fd0-439a-b64a-6cc54b56595c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:40.108556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5445a-afb4-40a3-9098-42266c11dd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:40.109301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5445a-afb4-40a3-9098-42266c11dd74 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:40.109585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5445a-afb4-40a3-9098-42266c11dd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:40.109865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5445a-afb4-40a3-9098-42266c11dd74 None admin] PATH_INFO: `/v3/projects/b461eb3e378d4dc897c1a8ad160b921c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:40.111030 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5445a-afb4-40a3-9098-42266c11dd74 None admin] RBAC: Authorizing `identity:get_project(project_id=b461eb3e378d4dc897c1a8ad160b921c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:40.112489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5445a-afb4-40a3-9098-42266c11dd74 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:40.113827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67a1a6-9637-40a0-b0d8-c5baf07e499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:40.114421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 40/81] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/projects/b461eb3e378d4dc897c1a8ad160b921c => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:40.114548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67a1a6-9637-40a0-b0d8-c5baf07e499d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:40.114757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67a1a6-9637-40a0-b0d8-c5baf07e499d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:40.115084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67a1a6-9637-40a0-b0d8-c5baf07e499d None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:40.116560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67a1a6-9637-40a0-b0d8-c5baf07e499d None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:40.117982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67a1a6-9637-40a0-b0d8-c5baf07e499d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:40.119270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5445a-afb4-40a3-9098-42266c11dd74 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:40.119754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 42/82] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:39 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:40.122373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67a1a6-9637-40a0-b0d8-c5baf07e499d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:40.228157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67a1a6-9637-40a0-b0d8-c5baf07e499d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:40.228655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5445a-afb4-40a3-9098-42266c11dd74 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:40.265763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0715a83-fc83-4055-aaea-3efc390082a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:40.266406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0715a83-fc83-4055-aaea-3efc390082a2 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:40.266706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0715a83-fc83-4055-aaea-3efc390082a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:40.266969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0715a83-fc83-4055-aaea-3efc390082a2 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:40.267637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0715a83-fc83-4055-aaea-3efc390082a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:40.268401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77960a03-9bbb-4c37-8eb4-39787c66dab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:40.269074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0715a83-fc83-4055-aaea-3efc390082a2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:40.269374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77960a03-9bbb-4c37-8eb4-39787c66dab1 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:40.269792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77960a03-9bbb-4c37-8eb4-39787c66dab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:40.270213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77960a03-9bbb-4c37-8eb4-39787c66dab1 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/users/10a6976357d0445a9fa2891d68878d0a/roles/eaef1c14dd23498185dccc05b242ff6a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:40.272604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77960a03-9bbb-4c37-8eb4-39787c66dab1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8434c7cd8ed4e18a86583801f3af1f7, user_id=10a6976357d0445a9fa2891d68878d0a, role_id=eaef1c14dd23498185dccc05b242ff6a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:40.274190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77960a03-9bbb-4c37-8eb4-39787c66dab1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:40.276549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:40.289287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 41/83] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:40 2022] PUT /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/users/10a6976357d0445a9fa2891d68878d0a/roles/eaef1c14dd23498185dccc05b242ff6a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:11:40.294763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77960a03-9bbb-4c37-8eb4-39787c66dab1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:40.294846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 43/84] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:11:40 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:40.299433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0715a83-fc83-4055-aaea-3efc390082a2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:40.339395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:40.483256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77960a03-9bbb-4c37-8eb4-39787c66dab1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:40.531156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0715a83-fc83-4055-aaea-3efc390082a2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:40.542829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d59ef5-325e-4614-9d89-66dcc106d648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:40.543749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59ef5-325e-4614-9d89-66dcc106d648 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:40.544321 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59ef5-325e-4614-9d89-66dcc106d648 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:40.544613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59ef5-325e-4614-9d89-66dcc106d648 None admin] PATH_INFO: `/v3/projects/b461eb3e378d4dc897c1a8ad160b921c/users/10a6976357d0445a9fa2891d68878d0a/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:40.546796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d59ef5-325e-4614-9d89-66dcc106d648 None admin] RBAC: Authorizing `identity:create_grant(project_id=b461eb3e378d4dc897c1a8ad160b921c, user_id=10a6976357d0445a9fa2891d68878d0a, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:40.548691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d59ef5-325e-4614-9d89-66dcc106d648 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:40.564204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 42/85] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:40 2022] PUT /identity/v3/projects/b461eb3e378d4dc897c1a8ad160b921c/users/10a6976357d0445a9fa2891d68878d0a/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:11:40.570966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b74a8-078d-492c-b2d9-ea286807ecd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:40.572093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b74a8-078d-492c-b2d9-ea286807ecd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:40.572603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b74a8-078d-492c-b2d9-ea286807ecd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:40.573082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b74a8-078d-492c-b2d9-ea286807ecd0 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/users/e9716bb37d07416391c7fe0f922727e4/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:40.576171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b74a8-078d-492c-b2d9-ea286807ecd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8434c7cd8ed4e18a86583801f3af1f7, user_id=e9716bb37d07416391c7fe0f922727e4, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:40.578025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b74a8-078d-492c-b2d9-ea286807ecd0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:40.601563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 44/86] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:40 2022] PUT /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/users/e9716bb37d07416391c7fe0f922727e4/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:11:42.340481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2110694-7498-49d8-bbf7-476f88436fd1 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:42.340965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2110694-7498-49d8-bbf7-476f88436fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:42.340965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2110694-7498-49d8-bbf7-476f88436fd1 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:42.342272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 43/87] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:42.347773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44f9d-b64f-413d-a72d-895083b1ceb1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:42.348566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44f9d-b64f-413d-a72d-895083b1ceb1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:42.349046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44f9d-b64f-413d-a72d-895083b1ceb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:42.367992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e4d44f9d-b64f-413d-a72d-895083b1ceb1 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:42.369472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e4d44f9d-b64f-413d-a72d-895083b1ceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:42.451636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 45/88] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:42.993871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2110694-7498-49d8-bbf7-476f88436fd1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:43.061065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:43.122591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2110694-7498-49d8-bbf7-476f88436fd1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:43.176449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9fa10-6583-48d7-9cd6-7b2babc8b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:43.177502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fa10-6583-48d7-9cd6-7b2babc8b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.178520 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fa10-6583-48d7-9cd6-7b2babc8b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.178995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fa10-6583-48d7-9cd6-7b2babc8b1f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.194203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9fa10-6583-48d7-9cd6-7b2babc8b1f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-72c9fa10-6583-48d7-9cd6-7b2babc8b1f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:43.199760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:43.200613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:43.201379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:43.203960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 44/89] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:43.211428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44f9d-b64f-413d-a72d-895083b1ceb1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:43.213605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:43.362004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44f9d-b64f-413d-a72d-895083b1ceb1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:43.405977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84aedf-1695-49eb-b697-68d867b0d14a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.406986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84aedf-1695-49eb-b697-68d867b0d14a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.407453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84aedf-1695-49eb-b697-68d867b0d14a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.408801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 45/90] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:43.414175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba819f4-ce36-4703-9bcd-c720ade7d63d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.414547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba819f4-ce36-4703-9bcd-c720ade7d63d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.414827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba819f4-ce36-4703-9bcd-c720ade7d63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.416755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:43.417581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.418070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.418554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.419413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:43.419918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:43.421535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:43.433844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 46/91] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:43.440167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:43.446608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3ba819f4-ce36-4703-9bcd-c720ade7d63d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:43.447867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3ba819f4-ce36-4703-9bcd-c720ade7d63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:43.535654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:43.573318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 46/92] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:43.574970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298848-12da-408f-8d66-4916c4cee338 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.575448 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298848-12da-408f-8d66-4916c4cee338 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.575910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298848-12da-408f-8d66-4916c4cee338 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.577336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 47/93] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:43.598955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c42324-5310-45c5-af8b-e8679779b041 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:43.600958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16834f-4625-4588-a5f0-68641e894424 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.601517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16834f-4625-4588-a5f0-68641e894424 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.601984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16834f-4625-4588-a5f0-68641e894424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.634451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3f16834f-4625-4588-a5f0-68641e894424 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:43.634451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3f16834f-4625-4588-a5f0-68641e894424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:43.641660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260304a-1eca-4d5f-8d0b-893a9a861482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:43.642307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260304a-1eca-4d5f-8d0b-893a9a861482 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.642621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260304a-1eca-4d5f-8d0b-893a9a861482 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.642892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260304a-1eca-4d5f-8d0b-893a9a861482 None admin] PATH_INFO: `/v3/users/10a6976357d0445a9fa2891d68878d0a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.647316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260304a-1eca-4d5f-8d0b-893a9a861482 None admin] RBAC: Authorizing `identity:get_user(user_id=10a6976357d0445a9fa2891d68878d0a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:43.648843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260304a-1eca-4d5f-8d0b-893a9a861482 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:43.650506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 47/94] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:43 2022] GET /identity/v3/users/10a6976357d0445a9fa2891d68878d0a => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:43.655965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260304a-1eca-4d5f-8d0b-893a9a861482 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:43.723713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:43.782761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 48/95] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:43.863776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260304a-1eca-4d5f-8d0b-893a9a861482 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:43.890074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a392-403c-4167-8618-9b9d4600755e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.890370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a392-403c-4167-8618-9b9d4600755e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.890593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a392-403c-4167-8618-9b9d4600755e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.892059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 49/96] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:11:43.897161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b6c36-b3d6-411f-a714-55d09b401825 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.897625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b6c36-b3d6-411f-a714-55d09b401825 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.897906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b6c36-b3d6-411f-a714-55d09b401825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:43.939963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-107b6c36-b3d6-411f-a714-55d09b401825 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:43.940610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:43.941452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-107b6c36-b3d6-411f-a714-55d09b401825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:43.959220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefecc5-1b4a-4c52-989d-af4128b63810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:43.959858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefecc5-1b4a-4c52-989d-af4128b63810 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:43.960137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefecc5-1b4a-4c52-989d-af4128b63810 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:43.960410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefecc5-1b4a-4c52-989d-af4128b63810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.046675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 50/97] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:44.063104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b6c36-b3d6-411f-a714-55d09b401825 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.099779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefecc5-1b4a-4c52-989d-af4128b63810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:44.100629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefecc5-1b4a-4c52-989d-af4128b63810 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:44.232012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b6c36-b3d6-411f-a714-55d09b401825 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:44.234381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:44.250309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 48/98] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:11:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 10:11:44.251521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba7c28-5a99-4201-8291-11f0687c0cfc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.251806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba7c28-5a99-4201-8291-11f0687c0cfc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.252082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba7c28-5a99-4201-8291-11f0687c0cfc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.253466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 49/99] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:44.254033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba7c28-5a99-4201-8291-11f0687c0cfc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.275767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc51bd-e15b-45bd-9e15-138c83fcfde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:44.276409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc51bd-e15b-45bd-9e15-138c83fcfde8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.276847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc51bd-e15b-45bd-9e15-138c83fcfde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.277126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc51bd-e15b-45bd-9e15-138c83fcfde8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.292711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc51bd-e15b-45bd-9e15-138c83fcfde8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:44.295337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-bacc51bd-e15b-45bd-9e15-138c83fcfde8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:44.297970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:44.299068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 51/100] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:44.300003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc51bd-e15b-45bd-9e15-138c83fcfde8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.373879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba7c28-5a99-4201-8291-11f0687c0cfc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:44.404559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc51bd-e15b-45bd-9e15-138c83fcfde8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:44.408196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576fcf0-d3c6-411b-bb76-91f3d8ba32ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:44.408827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576fcf0-d3c6-411b-bb76-91f3d8ba32ee None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.409107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576fcf0-d3c6-411b-bb76-91f3d8ba32ee None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.409379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576fcf0-d3c6-411b-bb76-91f3d8ba32ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.420111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576fcf0-d3c6-411b-bb76-91f3d8ba32ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f576fcf0-d3c6-411b-bb76-91f3d8ba32ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:44.424952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:44.425992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:44.426838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 50/101] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:44.427810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6f47-a119-4601-815b-2ace0f438515 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.428120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6f47-a119-4601-815b-2ace0f438515 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.428389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6f47-a119-4601-815b-2ace0f438515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.441788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b38cf7-b2b2-43f1-8da7-da4f819032a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:44.442456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38cf7-b2b2-43f1-8da7-da4f819032a1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.442741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38cf7-b2b2-43f1-8da7-da4f819032a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.443066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38cf7-b2b2-43f1-8da7-da4f819032a1 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.444302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b38cf7-b2b2-43f1-8da7-da4f819032a1 None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:44.445180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-68af6f47-a119-4601-815b-2ace0f438515 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:44.445748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b38cf7-b2b2-43f1-8da7-da4f819032a1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:44.446395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-68af6f47-a119-4601-815b-2ace0f438515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:44.447422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 52/102] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:44.448599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77249fb-d41c-403f-a49a-2ef61d2ecadc None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.448885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77249fb-d41c-403f-a49a-2ef61d2ecadc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.449158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77249fb-d41c-403f-a49a-2ef61d2ecadc None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.450375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 53/103] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:44.455118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77249fb-d41c-403f-a49a-2ef61d2ecadc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.488542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:44.522179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 51/104] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:44.526293 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6f47-a119-4601-815b-2ace0f438515 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.555515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:44.561271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77249fb-d41c-403f-a49a-2ef61d2ecadc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:44.597962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:44.598665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.598944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.599255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.600398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:44.600710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:44.602173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:44.614785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 54/105] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:44.614785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.645619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6f47-a119-4601-815b-2ace0f438515 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:44.680496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bed5c-b031-4fd8-862a-cd2e8bf78462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:44.681277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bed5c-b031-4fd8-862a-cd2e8bf78462 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.681508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bed5c-b031-4fd8-862a-cd2e8bf78462 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.681811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bed5c-b031-4fd8-862a-cd2e8bf78462 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.696675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bed5c-b031-4fd8-862a-cd2e8bf78462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9c8bed5c-b031-4fd8-862a-cd2e8bf78462 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:44.701947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:44.702858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:44.703713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 52/106] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:44.706217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bed5c-b031-4fd8-862a-cd2e8bf78462 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.717292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:44.725200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003ed3-7cd5-4840-8c47-c14a685c7794 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:44.763937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc253124-4835-4f66-9578-fb562b719f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:44.764367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc253124-4835-4f66-9578-fb562b719f35 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.764645 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc253124-4835-4f66-9578-fb562b719f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.764908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc253124-4835-4f66-9578-fb562b719f35 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.765573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc253124-4835-4f66-9578-fb562b719f35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:44.765881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc253124-4835-4f66-9578-fb562b719f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:44.767283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc253124-4835-4f66-9578-fb562b719f35 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:44.775632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 55/107] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:44.781164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05186-7501-469b-9819-22e9940aa14f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.781784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05186-7501-469b-9819-22e9940aa14f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.781784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05186-7501-469b-9819-22e9940aa14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.797675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:44.800283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e2a05186-7501-469b-9819-22e9940aa14f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:44.801467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2a05186-7501-469b-9819-22e9940aa14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:44.815805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bed5c-b031-4fd8-862a-cd2e8bf78462 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:44.850066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:44.850701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:44.850979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:44.851252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:44.851974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] RBAC: Adding query filter params (role.id=cbe9824e14834e26a81ef6a408bd1bd4, scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7, user.id=10a6976357d0445a9fa2891d68878d0a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:44.852286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:44.853725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:44.861602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 53/108] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/role_assignments?user.id=10a6976357d0445a9fa2891d68878d0a&scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7&role.id=cbe9824e14834e26a81ef6a408bd1bd4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:44.862433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.883537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 56/109] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:44.887368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a05186-7501-469b-9819-22e9940aa14f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:44.978033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8b825-3fbf-4f6b-ae88-baff5ed2cc42 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:44.994211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.014421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a05186-7501-469b-9819-22e9940aa14f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.016289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a485a4-9275-4db6-82d2-6c18bc02370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.016943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a485a4-9275-4db6-82d2-6c18bc02370c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.017223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a485a4-9275-4db6-82d2-6c18bc02370c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.017498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a485a4-9275-4db6-82d2-6c18bc02370c None admin] PATH_INFO: `/v3/users/10a6976357d0445a9fa2891d68878d0a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.019125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a485a4-9275-4db6-82d2-6c18bc02370c None admin] RBAC: Authorizing `identity:get_user(user_id=10a6976357d0445a9fa2891d68878d0a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.020258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a485a4-9275-4db6-82d2-6c18bc02370c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.022662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 54/110] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/users/10a6976357d0445a9fa2891d68878d0a => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:45.022662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a485a4-9275-4db6-82d2-6c18bc02370c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.048512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.071924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.071924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.072305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.074850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.074850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:45.077699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.077699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.093834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 57/111] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:45.094728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.158790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a485a4-9275-4db6-82d2-6c18bc02370c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.191640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dad8c-6cb2-4a68-aa17-29f85c3f178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.192258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dad8c-6cb2-4a68-aa17-29f85c3f178b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.192533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dad8c-6cb2-4a68-aa17-29f85c3f178b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.192809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dad8c-6cb2-4a68-aa17-29f85c3f178b None admin] PATH_INFO: `/v3/users/10a6976357d0445a9fa2891d68878d0a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.194567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68dad8c-6cb2-4a68-aa17-29f85c3f178b None admin] RBAC: Authorizing `identity:get_user(user_id=10a6976357d0445a9fa2891d68878d0a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.196169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68dad8c-6cb2-4a68-aa17-29f85c3f178b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.197740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 55/112] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/users/10a6976357d0445a9fa2891d68878d0a => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:45.198525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dad8c-6cb2-4a68-aa17-29f85c3f178b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.214227 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6109a3-5c36-466d-b03a-47e81af95084 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.247075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd40f8-dc6a-4b16-a888-77b649927f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.247706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd40f8-dc6a-4b16-a888-77b649927f2b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.247984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd40f8-dc6a-4b16-a888-77b649927f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.248283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd40f8-dc6a-4b16-a888-77b649927f2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.249378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd40f8-dc6a-4b16-a888-77b649927f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.250886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd40f8-dc6a-4b16-a888-77b649927f2b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.252440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 58/113] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:45.257332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd40f8-dc6a-4b16-a888-77b649927f2b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.273129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.279016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.298144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dad8c-6cb2-4a68-aa17-29f85c3f178b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.330804 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30a26f-1aef-4799-b918-c14d2a6dd721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.331404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30a26f-1aef-4799-b918-c14d2a6dd721 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.331683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30a26f-1aef-4799-b918-c14d2a6dd721 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.331954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30a26f-1aef-4799-b918-c14d2a6dd721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.355629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd40f8-dc6a-4b16-a888-77b649927f2b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.388162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34936a2e-bf21-42e5-ac5f-3e8d414b5303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.388811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34936a2e-bf21-42e5-ac5f-3e8d414b5303 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.389088 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34936a2e-bf21-42e5-ac5f-3e8d414b5303 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.389363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34936a2e-bf21-42e5-ac5f-3e8d414b5303 None admin] PATH_INFO: `/v3/users/864c2d793b3f4384859b141d854d8026` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.398252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30a26f-1aef-4799-b918-c14d2a6dd721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.399864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30a26f-1aef-4799-b918-c14d2a6dd721 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.405619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34936a2e-bf21-42e5-ac5f-3e8d414b5303 None admin] RBAC: Authorizing `identity:get_user(user_id=864c2d793b3f4384859b141d854d8026)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.407092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34936a2e-bf21-42e5-ac5f-3e8d414b5303 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.408727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 59/114] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/users/864c2d793b3f4384859b141d854d8026 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:45.409489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34936a2e-bf21-42e5-ac5f-3e8d414b5303 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.479396 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.487184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.500034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 56/115] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:11:45.500866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30a26f-1aef-4799-b918-c14d2a6dd721 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.513049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34936a2e-bf21-42e5-ac5f-3e8d414b5303 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.545698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fd43b-be7f-4b1e-8778-a5e6c05beb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.546257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fd43b-be7f-4b1e-8778-a5e6c05beb0f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.546557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fd43b-be7f-4b1e-8778-a5e6c05beb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.546827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fd43b-be7f-4b1e-8778-a5e6c05beb0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.556621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fd43b-be7f-4b1e-8778-a5e6c05beb0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b86fd43b-be7f-4b1e-8778-a5e6c05beb0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:45.561791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:45.562874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 60/116] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:45.563628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fd43b-be7f-4b1e-8778-a5e6c05beb0f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.603100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30a26f-1aef-4799-b918-c14d2a6dd721 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.635904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4487695-ffdb-428d-847b-9406fb162cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.636507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4487695-ffdb-428d-847b-9406fb162cee None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.636785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4487695-ffdb-428d-847b-9406fb162cee None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.637054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4487695-ffdb-428d-847b-9406fb162cee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.660479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fd43b-be7f-4b1e-8778-a5e6c05beb0f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.685636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.692558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abeb105-aac9-4f9b-9bba-c77f871e17f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.694980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb105-aac9-4f9b-9bba-c77f871e17f3 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.695265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb105-aac9-4f9b-9bba-c77f871e17f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.695541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb105-aac9-4f9b-9bba-c77f871e17f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.696195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abeb105-aac9-4f9b-9bba-c77f871e17f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.697658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abeb105-aac9-4f9b-9bba-c77f871e17f3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.702601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.704954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4487695-ffdb-428d-847b-9406fb162cee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.706443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4487695-ffdb-428d-847b-9406fb162cee None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.727052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 61/117] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:11:45 2022] POST /identity/v3/projects => generated 302 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:45.727706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abeb105-aac9-4f9b-9bba-c77f871e17f3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.820464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 57/118] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:11:45.821135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4487695-ffdb-428d-847b-9406fb162cee None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.840590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abeb105-aac9-4f9b-9bba-c77f871e17f3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.885780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcbe1f-4336-4ccb-bf79-aeb51d033179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.886471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcbe1f-4336-4ccb-bf79-aeb51d033179 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.886787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcbe1f-4336-4ccb-bf79-aeb51d033179 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.887018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcbe1f-4336-4ccb-bf79-aeb51d033179 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.888369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcbe1f-4336-4ccb-bf79-aeb51d033179 None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.889929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcbe1f-4336-4ccb-bf79-aeb51d033179 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.896703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 62/119] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:45.896703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcbe1f-4336-4ccb-bf79-aeb51d033179 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:45.920363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.944360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4487695-ffdb-428d-847b-9406fb162cee None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:45.966430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:45.982478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72676df-a118-4723-988c-032daeb98718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:45.983405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72676df-a118-4723-988c-032daeb98718 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:45.983858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72676df-a118-4723-988c-032daeb98718 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:45.984327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72676df-a118-4723-988c-032daeb98718 None admin] PATH_INFO: `/v3/projects/b461eb3e378d4dc897c1a8ad160b921c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:45.986400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72676df-a118-4723-988c-032daeb98718 None admin] RBAC: Authorizing `identity:get_project(project_id=b461eb3e378d4dc897c1a8ad160b921c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:45.988084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72676df-a118-4723-988c-032daeb98718 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:45.990405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 58/120] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/projects/b461eb3e378d4dc897c1a8ad160b921c => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:45.995299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72676df-a118-4723-988c-032daeb98718 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:46.018729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcbe1f-4336-4ccb-bf79-aeb51d033179 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:46.067660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:46.068898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:46.069178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:46.069452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:46.070129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:46.070456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:46.071917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:46.086389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 63/121] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:46.087517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:46.107731 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72676df-a118-4723-988c-032daeb98718 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:46.145367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1603a4d-f426-469e-8a6a-1b467ea3e6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:46.145998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1603a4d-f426-469e-8a6a-1b467ea3e6bf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:46.146889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1603a4d-f426-469e-8a6a-1b467ea3e6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:46.147170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1603a4d-f426-469e-8a6a-1b467ea3e6bf None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:46.148306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1603a4d-f426-469e-8a6a-1b467ea3e6bf None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:46.149745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1603a4d-f426-469e-8a6a-1b467ea3e6bf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:46.151404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 59/122] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:45 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:46.152164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1603a4d-f426-469e-8a6a-1b467ea3e6bf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:46.178761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:46.200804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd276951-a82d-4280-b8db-82bf1cc845f2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:46.218614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:46.239095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eace45-755a-4b52-a3d4-cbe498f50478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:46.240864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eace45-755a-4b52-a3d4-cbe498f50478 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:46.241162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eace45-755a-4b52-a3d4-cbe498f50478 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:46.241439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eace45-755a-4b52-a3d4-cbe498f50478 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/users/864c2d793b3f4384859b141d854d8026/roles/c1cacdb19cc543428839254e4ecf77a2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:46.243831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eace45-755a-4b52-a3d4-cbe498f50478 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8434c7cd8ed4e18a86583801f3af1f7, user_id=864c2d793b3f4384859b141d854d8026, role_id=c1cacdb19cc543428839254e4ecf77a2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:46.245633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eace45-755a-4b52-a3d4-cbe498f50478 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:46.266224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 64/123] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:46 2022] PUT /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/users/864c2d793b3f4384859b141d854d8026/roles/c1cacdb19cc543428839254e4ecf77a2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:11:46.266224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eace45-755a-4b52-a3d4-cbe498f50478 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:46.276025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1603a4d-f426-469e-8a6a-1b467ea3e6bf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:46.317829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:46.319475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:46.320857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:46.322016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:46.322863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] RBAC: Adding query filter params (role.id=cbe9824e14834e26a81ef6a408bd1bd4, scope.project.id=b461eb3e378d4dc897c1a8ad160b921c, user.id=10a6976357d0445a9fa2891d68878d0a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:46.323251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:46.324922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:46.332026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 60/124] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:11:46 2022] GET /identity/v3/role_assignments?user.id=10a6976357d0445a9fa2891d68878d0a&scope.project.id=b461eb3e378d4dc897c1a8ad160b921c&role.id=cbe9824e14834e26a81ef6a408bd1bd4 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:46.333039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:46.433676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:46.457514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73674f26-27e8-4461-b059-1f1d6393afdf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:46.468886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eace45-755a-4b52-a3d4-cbe498f50478 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:46.504636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:46.505666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:46.506132 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:46.506616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:46.507598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] RBAC: Adding query filter params (role.id=eaef1c14dd23498185dccc05b242ff6a, scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7, user.id=10a6976357d0445a9fa2891d68878d0a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:46.508139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:46.509818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:46.518571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 61/125] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:11:46 2022] GET /identity/v3/role_assignments?user.id=10a6976357d0445a9fa2891d68878d0a&scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7&role.id=eaef1c14dd23498185dccc05b242ff6a => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:46.539085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2d1a6-b522-46fd-b2f4-7bf8b4df29b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:46.540529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d1a6-b522-46fd-b2f4-7bf8b4df29b7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:46.541115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d1a6-b522-46fd-b2f4-7bf8b4df29b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:46.541622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d1a6-b522-46fd-b2f4-7bf8b4df29b7 None admin] PATH_INFO: `/v3/users/e9716bb37d07416391c7fe0f922727e4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:46.543263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2d1a6-b522-46fd-b2f4-7bf8b4df29b7 None admin] RBAC: Authorizing `identity:get_user(user_id=e9716bb37d07416391c7fe0f922727e4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:46.545040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2d1a6-b522-46fd-b2f4-7bf8b4df29b7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:46.547287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 65/126] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:46 2022] GET /identity/v3/users/e9716bb37d07416391c7fe0f922727e4 => generated 247 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:46.553917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:46.673195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204b26-9ba0-46fc-b59d-7294712a02f3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:46.722376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eeb81-b24a-4996-8087-c45e5271b500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:46.723463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eeb81-b24a-4996-8087-c45e5271b500 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:46.723747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eeb81-b24a-4996-8087-c45e5271b500 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:46.724019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eeb81-b24a-4996-8087-c45e5271b500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:46.744149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:46.803709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eeb81-b24a-4996-8087-c45e5271b500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:46.805311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eeb81-b24a-4996-8087-c45e5271b500 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:46.908570 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 62/127] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:11:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:11:46.914857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2d1a6-b522-46fd-b2f4-7bf8b4df29b7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:46.963181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:47.020806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2d1a6-b522-46fd-b2f4-7bf8b4df29b7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:47.055833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89250212-da8e-470c-9a61-ee3328deb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:47.056469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89250212-da8e-470c-9a61-ee3328deb685 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:47.056752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89250212-da8e-470c-9a61-ee3328deb685 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:47.057027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89250212-da8e-470c-9a61-ee3328deb685 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:47.058154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89250212-da8e-470c-9a61-ee3328deb685 None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:47.059633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89250212-da8e-470c-9a61-ee3328deb685 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:47.061429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 66/128] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:46 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:47.067086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eeb81-b24a-4996-8087-c45e5271b500 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:47.117053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:47.173183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eeb81-b24a-4996-8087-c45e5271b500 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:47.208093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562e8ad-a647-4b16-b2a3-ac822837c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:47.208793 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562e8ad-a647-4b16-b2a3-ac822837c4ec None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:47.209079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562e8ad-a647-4b16-b2a3-ac822837c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:47.209359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562e8ad-a647-4b16-b2a3-ac822837c4ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:47.210091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562e8ad-a647-4b16-b2a3-ac822837c4ec None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7, user.id=e9716bb37d07416391c7fe0f922727e4) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:47.210432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562e8ad-a647-4b16-b2a3-ac822837c4ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:47.211864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562e8ad-a647-4b16-b2a3-ac822837c4ec None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:47.217867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 63/129] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:11:47 2022] GET /identity/v3/role_assignments?user.id=e9716bb37d07416391c7fe0f922727e4&scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:47.980004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a658b9-1f51-4a33-86e7-dc5e2ca6feda None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:47.980521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a658b9-1f51-4a33-86e7-dc5e2ca6feda None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:47.980603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a658b9-1f51-4a33-86e7-dc5e2ca6feda None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:47.981874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 67/130] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:47.987134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956afd8-d17d-40df-a021-4b959056f3ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:47.987560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956afd8-d17d-40df-a021-4b959056f3ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:47.987682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956afd8-d17d-40df-a021-4b959056f3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:48.007003 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a956afd8-d17d-40df-a021-4b959056f3ad None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:48.008167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a956afd8-d17d-40df-a021-4b959056f3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:48.077871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 64/131] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:48.349703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a658b9-1f51-4a33-86e7-dc5e2ca6feda None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:48.454294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a658b9-1f51-4a33-86e7-dc5e2ca6feda None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:48.467472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:48.491130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be5957-8357-429d-96c9-4b79710e389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:48.491757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5957-8357-429d-96c9-4b79710e389b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:48.492035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5957-8357-429d-96c9-4b79710e389b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:48.492305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5957-8357-429d-96c9-4b79710e389b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:48.493389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be5957-8357-429d-96c9-4b79710e389b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:48.494981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be5957-8357-429d-96c9-4b79710e389b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:48.496549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 68/132] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:48.502145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956afd8-d17d-40df-a021-4b959056f3ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:48.545000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:48.603262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956afd8-d17d-40df-a021-4b959056f3ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:48.636596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addb037-15aa-431d-9d8d-73d86721df3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:48.637197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb037-15aa-431d-9d8d-73d86721df3e None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:48.637479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb037-15aa-431d-9d8d-73d86721df3e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:48.637755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb037-15aa-431d-9d8d-73d86721df3e None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:48.638443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addb037-15aa-431d-9d8d-73d86721df3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:48.639883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addb037-15aa-431d-9d8d-73d86721df3e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:48.652558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbd9ef-ebe2-46c5-9bba-2354efbb1ddb None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:48.652903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbd9ef-ebe2-46c5-9bba-2354efbb1ddb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:48.653241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbd9ef-ebe2-46c5-9bba-2354efbb1ddb None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:48.654854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 69/133] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:48.660139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9c34-3d8f-48b3-a1a8-25387df9fc93 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:48.660432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9c34-3d8f-48b3-a1a8-25387df9fc93 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:48.660702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9c34-3d8f-48b3-a1a8-25387df9fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:48.661066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 65/134] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:11:48 2022] POST /identity/v3/users => generated 283 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:11:48.682430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fb3e9c34-3d8f-48b3-a1a8-25387df9fc93 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:48.683660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb3e9c34-3d8f-48b3-a1a8-25387df9fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:48.757471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 70/135] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:49.032103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addb037-15aa-431d-9d8d-73d86721df3e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:49.140214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:49.146047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addb037-15aa-431d-9d8d-73d86721df3e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:49.183741 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585edafc-c0db-4d40-87cf-67830b3a78c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:49.184745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585edafc-c0db-4d40-87cf-67830b3a78c3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:49.185241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585edafc-c0db-4d40-87cf-67830b3a78c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:49.185715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585edafc-c0db-4d40-87cf-67830b3a78c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:49.196662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585edafc-c0db-4d40-87cf-67830b3a78c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-585edafc-c0db-4d40-87cf-67830b3a78c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:49.202281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:49.203207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:11:49.204083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:49.206376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 66/136] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:49.218067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e9c34-3d8f-48b3-a1a8-25387df9fc93 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:49.317237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e9c34-3d8f-48b3-a1a8-25387df9fc93 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:49.328964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:49.359066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:49.359687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:49.359971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:49.360246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:49.360919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:49.361222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:49.362634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:49.371513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 71/137] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:49.376776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585edafc-c0db-4d40-87cf-67830b3a78c3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:49.486089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585edafc-c0db-4d40-87cf-67830b3a78c3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:49.526218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52478778-594d-4015-a0e1-de3dd68be272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:49.526897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52478778-594d-4015-a0e1-de3dd68be272 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:49.527174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52478778-594d-4015-a0e1-de3dd68be272 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:49.527446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52478778-594d-4015-a0e1-de3dd68be272 None admin] PATH_INFO: `/v3/users/864c2d793b3f4384859b141d854d8026` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:49.528599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52478778-594d-4015-a0e1-de3dd68be272 None admin] RBAC: Authorizing `identity:get_user(user_id=864c2d793b3f4384859b141d854d8026)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:49.530036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52478778-594d-4015-a0e1-de3dd68be272 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:49.531885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 67/138] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:49 2022] GET /identity/v3/users/864c2d793b3f4384859b141d854d8026 => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:49.537691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:49.642766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bb0fb-00d9-455e-ae43-1a5ddfffbc35 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:49.682199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21380c6-c6e2-4cff-aa25-422cc383df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:49.682850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21380c6-c6e2-4cff-aa25-422cc383df55 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:49.683130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21380c6-c6e2-4cff-aa25-422cc383df55 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:49.683406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21380c6-c6e2-4cff-aa25-422cc383df55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:49.729518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:49.758911 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21380c6-c6e2-4cff-aa25-422cc383df55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:49.760391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21380c6-c6e2-4cff-aa25-422cc383df55 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:49.866747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 72/139] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:11:49.872530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52478778-594d-4015-a0e1-de3dd68be272 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:49.901310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:49.984870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52478778-594d-4015-a0e1-de3dd68be272 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:50.018751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2776f2-3bcb-42c0-bb3c-ecb3275881df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:50.019377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2776f2-3bcb-42c0-bb3c-ecb3275881df None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:50.019658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2776f2-3bcb-42c0-bb3c-ecb3275881df None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:50.019934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2776f2-3bcb-42c0-bb3c-ecb3275881df None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:50.021152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2776f2-3bcb-42c0-bb3c-ecb3275881df None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:50.022679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2776f2-3bcb-42c0-bb3c-ecb3275881df None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:50.024604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 68/140] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:49 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:50.030307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21380c6-c6e2-4cff-aa25-422cc383df55 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:50.106584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:50.139042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21380c6-c6e2-4cff-aa25-422cc383df55 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:50.174945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f7c96-4b3f-4a85-b147-016e8f028ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:50.175477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f7c96-4b3f-4a85-b147-016e8f028ccf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:50.175752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f7c96-4b3f-4a85-b147-016e8f028ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:50.176020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f7c96-4b3f-4a85-b147-016e8f028ccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:50.176746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f7c96-4b3f-4a85-b147-016e8f028ccf None admin] RBAC: Adding query filter params (role.id=c1cacdb19cc543428839254e4ecf77a2, scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7, user.id=864c2d793b3f4384859b141d854d8026) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:50.177052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f7c96-4b3f-4a85-b147-016e8f028ccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:50.178752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f7c96-4b3f-4a85-b147-016e8f028ccf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:50.190467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 73/141] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:11:50 2022] GET /identity/v3/role_assignments?user.id=864c2d793b3f4384859b141d854d8026&scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7&role.id=c1cacdb19cc543428839254e4ecf77a2 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:51.014987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa7b61-b7ff-4033-a142-d0ed2d581ec8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.015555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa7b61-b7ff-4033-a142-d0ed2d581ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.015555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa7b61-b7ff-4033-a142-d0ed2d581ec8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.017166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 69/142] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:51.023250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ab6b-8274-4b4f-8921-0a4bcc0d299a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.023789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ab6b-8274-4b4f-8921-0a4bcc0d299a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.023789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ab6b-8274-4b4f-8921-0a4bcc0d299a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.043582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3662ab6b-8274-4b4f-8921-0a4bcc0d299a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:51.044794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3662ab6b-8274-4b4f-8921-0a4bcc0d299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:51.128577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 74/143] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:51.141737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e8c9-6fe5-45de-8783-a076d08951d5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.142026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e8c9-6fe5-45de-8783-a076d08951d5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.142303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e8c9-6fe5-45de-8783-a076d08951d5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.143786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 70/144] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:51.149263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e3852-ceb0-4157-9ef6-89190a60262d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.149547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e3852-ceb0-4157-9ef6-89190a60262d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.149821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e3852-ceb0-4157-9ef6-89190a60262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.168208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-743e3852-ceb0-4157-9ef6-89190a60262d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:51.169376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-743e3852-ceb0-4157-9ef6-89190a60262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:51.195957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db51ed-5a15-40fe-9da4-cbc79f204254 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.196268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db51ed-5a15-40fe-9da4-cbc79f204254 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.196528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db51ed-5a15-40fe-9da4-cbc79f204254 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.198022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 71/145] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:11:51.202651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06162115-fc71-4898-83f4-34d3c4edfc20 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.202942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06162115-fc71-4898-83f4-34d3c4edfc20 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.203223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06162115-fc71-4898-83f4-34d3c4edfc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.221769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-06162115-fc71-4898-83f4-34d3c4edfc20 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:51.223681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-06162115-fc71-4898-83f4-34d3c4edfc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:51.230342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:51.239241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 75/146] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:51.294460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 72/147] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:51.398603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e3852-ceb0-4157-9ef6-89190a60262d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:51.498372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e3852-ceb0-4157-9ef6-89190a60262d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:51.505134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06162115-fc71-4898-83f4-34d3c4edfc20 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:51.529968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:51.535049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd14dbd-1a32-4e5e-bfaf-5cadca99c2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:51.535684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14dbd-1a32-4e5e-bfaf-5cadca99c2ce None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.535960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14dbd-1a32-4e5e-bfaf-5cadca99c2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.536228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14dbd-1a32-4e5e-bfaf-5cadca99c2ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.546429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd14dbd-1a32-4e5e-bfaf-5cadca99c2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3bd14dbd-1a32-4e5e-bfaf-5cadca99c2ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:51.551242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:51.552110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:51.553175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 76/148] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:11:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:51.558013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd14dbd-1a32-4e5e-bfaf-5cadca99c2ce None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:51.612640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06162115-fc71-4898-83f4-34d3c4edfc20 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:51.613871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:51.647295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0dc90-dde7-430c-9571-bcdd01d3cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:51.647931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0dc90-dde7-430c-9571-bcdd01d3cf2c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.648219 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0dc90-dde7-430c-9571-bcdd01d3cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.648496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0dc90-dde7-430c-9571-bcdd01d3cf2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.658443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0dc90-dde7-430c-9571-bcdd01d3cf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:51.659976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd14dbd-1a32-4e5e-bfaf-5cadca99c2ce None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-14a0dc90-dde7-430c-9571-bcdd01d3cf2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:51.663816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:51.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:51.665691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 73/149] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:11:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:11:51.666534 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0dc90-dde7-430c-9571-bcdd01d3cf2c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:51.696383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:51.696988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.697268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.697539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.698200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:51.698537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:51.699908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:51.707911 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 77/150] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:51.707911 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:51.752208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:51.770483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0dc90-dde7-430c-9571-bcdd01d3cf2c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:51.805004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe964a4e-8405-47cc-a508-ef643250e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:51.805552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964a4e-8405-47cc-a508-ef643250e416 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.805827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964a4e-8405-47cc-a508-ef643250e416 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.806089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964a4e-8405-47cc-a508-ef643250e416 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.807230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe964a4e-8405-47cc-a508-ef643250e416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:51.808671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe964a4e-8405-47cc-a508-ef643250e416 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:51.809271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f926c7-d810-4ffd-9332-38938e4cb759 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:51.810220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 74/151] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:51.811008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe964a4e-8405-47cc-a508-ef643250e416 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:51.826584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:51.845034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:51.845631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.845904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.846171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.847101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:51.847405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:51.848773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:51.857025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 78/152] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:11:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:51.857853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:51.909910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe964a4e-8405-47cc-a508-ef643250e416 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:51.943690 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4087fe-573d-417c-a6a9-953a517ff31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:51.944303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4087fe-573d-417c-a6a9-953a517ff31b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.944578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4087fe-573d-417c-a6a9-953a517ff31b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.944844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4087fe-573d-417c-a6a9-953a517ff31b None admin] PATH_INFO: `/v3/users/80ba9dacb6bd48e39aeca10439ce8d25` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.953372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21114ba0-eced-4d92-9b9f-4e29344a75aa None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:51.958049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4087fe-573d-417c-a6a9-953a517ff31b None admin] RBAC: Authorizing `identity:get_user(user_id=80ba9dacb6bd48e39aeca10439ce8d25)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:51.960112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4087fe-573d-417c-a6a9-953a517ff31b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:51.961703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 75/153] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:51 2022] GET /identity/v3/users/80ba9dacb6bd48e39aeca10439ce8d25 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:51.962477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4087fe-573d-417c-a6a9-953a517ff31b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:51.972768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:51.986963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4dce78-4752-4b9b-b071-2c41120f24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:51.987715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dce78-4752-4b9b-b071-2c41120f24dc None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:51.988318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dce78-4752-4b9b-b071-2c41120f24dc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:51.988590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dce78-4752-4b9b-b071-2c41120f24dc None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:51.989235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dce78-4752-4b9b-b071-2c41120f24dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:51.991317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dce78-4752-4b9b-b071-2c41120f24dc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:52.019862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 79/154] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:11:51 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:52.019862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4dce78-4752-4b9b-b071-2c41120f24dc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:52.061850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:52.079265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4087fe-573d-417c-a6a9-953a517ff31b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:52.123587 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700d12b-7c2e-4984-be69-56f676e01902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:52.124676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d12b-7c2e-4984-be69-56f676e01902 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:52.125052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d12b-7c2e-4984-be69-56f676e01902 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:52.125824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d12b-7c2e-4984-be69-56f676e01902 None admin] PATH_INFO: `/v3/users/80ba9dacb6bd48e39aeca10439ce8d25` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:52.127125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700d12b-7c2e-4984-be69-56f676e01902 None admin] RBAC: Authorizing `identity:get_user(user_id=80ba9dacb6bd48e39aeca10439ce8d25)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:52.129366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700d12b-7c2e-4984-be69-56f676e01902 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:52.136865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 76/155] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:51 2022] GET /identity/v3/users/80ba9dacb6bd48e39aeca10439ce8d25 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:52.137794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700d12b-7c2e-4984-be69-56f676e01902 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:52.143467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4dce78-4752-4b9b-b071-2c41120f24dc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:52.182344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93404ecf-ff2e-44ad-a530-651cbb0a5a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:52.183949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404ecf-ff2e-44ad-a530-651cbb0a5a79 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:52.184449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404ecf-ff2e-44ad-a530-651cbb0a5a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:52.184917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404ecf-ff2e-44ad-a530-651cbb0a5a79 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:52.186296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93404ecf-ff2e-44ad-a530-651cbb0a5a79 None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:52.187993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93404ecf-ff2e-44ad-a530-651cbb0a5a79 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:52.189826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 80/156] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:52 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:52.196186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93404ecf-ff2e-44ad-a530-651cbb0a5a79 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:52.235108 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:52.262211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700d12b-7c2e-4984-be69-56f676e01902 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:52.304082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c63eb-c18e-49e6-b53b-a48c738fcfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:52.305060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c63eb-c18e-49e6-b53b-a48c738fcfbf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:52.305060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c63eb-c18e-49e6-b53b-a48c738fcfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:52.305224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c63eb-c18e-49e6-b53b-a48c738fcfbf None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:52.307887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c63eb-c18e-49e6-b53b-a48c738fcfbf None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:52.310140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c63eb-c18e-49e6-b53b-a48c738fcfbf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:52.310648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 77/157] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:52 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:52.316870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c63eb-c18e-49e6-b53b-a48c738fcfbf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:52.318468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93404ecf-ff2e-44ad-a530-651cbb0a5a79 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:52.322045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:52.360245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014c54b-61e5-46c4-b54e-ee7444964951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:52.360917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014c54b-61e5-46c4-b54e-ee7444964951 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:52.361192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014c54b-61e5-46c4-b54e-ee7444964951 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:52.361459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014c54b-61e5-46c4-b54e-ee7444964951 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185/users/80ba9dacb6bd48e39aeca10439ce8d25/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:52.363667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014c54b-61e5-46c4-b54e-ee7444964951 None admin] RBAC: Authorizing `identity:create_grant(project_id=102d784de27145ad9641bd35a5ea7185, user_id=80ba9dacb6bd48e39aeca10439ce8d25, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:52.366095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014c54b-61e5-46c4-b54e-ee7444964951 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:52.381582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 81/158] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:52 2022] PUT /identity/v3/projects/102d784de27145ad9641bd35a5ea7185/users/80ba9dacb6bd48e39aeca10439ce8d25/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:11:52.443896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c63eb-c18e-49e6-b53b-a48c738fcfbf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:52.486237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c3895-faa9-4ad7-98d5-cffab10e410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:52.486928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c3895-faa9-4ad7-98d5-cffab10e410a None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:52.487203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c3895-faa9-4ad7-98d5-cffab10e410a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:52.487472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c3895-faa9-4ad7-98d5-cffab10e410a None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185/users/80ba9dacb6bd48e39aeca10439ce8d25/roles/eaef1c14dd23498185dccc05b242ff6a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:52.489887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c3895-faa9-4ad7-98d5-cffab10e410a None admin] RBAC: Authorizing `identity:create_grant(project_id=102d784de27145ad9641bd35a5ea7185, user_id=80ba9dacb6bd48e39aeca10439ce8d25, role_id=eaef1c14dd23498185dccc05b242ff6a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:52.491848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c3895-faa9-4ad7-98d5-cffab10e410a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:52.513300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 78/159] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:52 2022] PUT /identity/v3/projects/102d784de27145ad9641bd35a5ea7185/users/80ba9dacb6bd48e39aeca10439ce8d25/roles/eaef1c14dd23498185dccc05b242ff6a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:11:54.334483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a835f3-d790-4a16-89f8-87b0d1dce638 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.335487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a835f3-d790-4a16-89f8-87b0d1dce638 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.335982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a835f3-d790-4a16-89f8-87b0d1dce638 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.337589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 82/160] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:54.342997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1534c-fe36-444e-ac26-fc3d5d1bd41e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.343800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1534c-fe36-444e-ac26-fc3d5d1bd41e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.344280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1534c-fe36-444e-ac26-fc3d5d1bd41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.352268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:54.364321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7eb1534c-fe36-444e-ac26-fc3d5d1bd41e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:54.366856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7eb1534c-fe36-444e-ac26-fc3d5d1bd41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:54.439764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 79/161] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:54.508738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e34d5-bc3a-48d5-8f2e-05fcfa376310 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.509061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e34d5-bc3a-48d5-8f2e-05fcfa376310 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.509276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e34d5-bc3a-48d5-8f2e-05fcfa376310 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.510676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 83/162] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:54.516377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161ae7d-ae16-48e4-97a3-511a7a5b44c5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.516658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161ae7d-ae16-48e4-97a3-511a7a5b44c5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.516927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161ae7d-ae16-48e4-97a3-511a7a5b44c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.535590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b161ae7d-ae16-48e4-97a3-511a7a5b44c5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:54.537251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b161ae7d-ae16-48e4-97a3-511a7a5b44c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:54.611207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc245e9-88a3-4a57-8fd2-f74114aeb5ba None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.611632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc245e9-88a3-4a57-8fd2-f74114aeb5ba None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.612134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc245e9-88a3-4a57-8fd2-f74114aeb5ba None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.615110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 84/164] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:54.620827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 80/164] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:54.622219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd4ee-40a4-41e2-a08d-488932c9fe12 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.622538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd4ee-40a4-41e2-a08d-488932c9fe12 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.622818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd4ee-40a4-41e2-a08d-488932c9fe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.641548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-dbffd4ee-40a4-41e2-a08d-488932c9fe12 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:54.642750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dbffd4ee-40a4-41e2-a08d-488932c9fe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:54.712363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 85/165] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:54.714895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb13fc-2df7-488e-aeb2-fe56663488eb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.715176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb13fc-2df7-488e-aeb2-fe56663488eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.715444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb13fc-2df7-488e-aeb2-fe56663488eb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.717097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 81/166] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:54.722201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdf47f-1415-482e-9785-6d69528a3fea None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.722572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdf47f-1415-482e-9785-6d69528a3fea None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.722922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdf47f-1415-482e-9785-6d69528a3fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.726928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb13fc-2df7-488e-aeb2-fe56663488eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:54.745507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-48fdf47f-1415-482e-9785-6d69528a3fea None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:54.747257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-48fdf47f-1415-482e-9785-6d69528a3fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:54.799552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:54.827139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 86/167] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:54.847948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb13fc-2df7-488e-aeb2-fe56663488eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:54.883914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd3da1-cab0-4829-8a81-2e8e7680bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:54.884738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd3da1-cab0-4829-8a81-2e8e7680bec4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:54.885157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd3da1-cab0-4829-8a81-2e8e7680bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:54.885560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd3da1-cab0-4829-8a81-2e8e7680bec4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:54.896646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd3da1-cab0-4829-8a81-2e8e7680bec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-29dd3da1-cab0-4829-8a81-2e8e7680bec4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:54.901638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:54.902740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:54.903823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:54.905823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 82/168] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:54.911754 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdf47f-1415-482e-9785-6d69528a3fea None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:54.929859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd3da1-cab0-4829-8a81-2e8e7680bec4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:54.940339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.041719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdf47f-1415-482e-9785-6d69528a3fea None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.041911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd3da1-cab0-4829-8a81-2e8e7680bec4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.080345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.080581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.081546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.081910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.082417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.083086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:55.083387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.085329 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.088406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae1bd7-4fe7-4f41-a3b8-71255d12d114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.089040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae1bd7-4fe7-4f41-a3b8-71255d12d114 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.089318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae1bd7-4fe7-4f41-a3b8-71255d12d114 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.089590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae1bd7-4fe7-4f41-a3b8-71255d12d114 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.091286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae1bd7-4fe7-4f41-a3b8-71255d12d114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.093029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae1bd7-4fe7-4f41-a3b8-71255d12d114 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.094199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 87/169] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:55.099507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 83/170] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:55.099645 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.100910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae1bd7-4fe7-4f41-a3b8-71255d12d114 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.183987 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.199204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360c0bc-938d-4209-9a48-c477e53b753b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.199454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae1bd7-4fe7-4f41-a3b8-71255d12d114 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.231780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b5be0-375d-49b5-a3f4-be6223d6cf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.232392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5be0-375d-49b5-a3f4-be6223d6cf5e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.232663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5be0-375d-49b5-a3f4-be6223d6cf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.232926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5be0-375d-49b5-a3f4-be6223d6cf5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.233429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9b616-f0b4-4a67-b5b0-72a6bca233a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.234041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9b616-f0b4-4a67-b5b0-72a6bca233a3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.234312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9b616-f0b4-4a67-b5b0-72a6bca233a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.234608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9b616-f0b4-4a67-b5b0-72a6bca233a3 None admin] PATH_INFO: `/v3/users/1e28b9c83c4247b0ae467d0c826ee8e0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.244137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b5be0-375d-49b5-a3f4-be6223d6cf5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:55.248153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9b616-f0b4-4a67-b5b0-72a6bca233a3 None admin] RBAC: Authorizing `identity:get_user(user_id=1e28b9c83c4247b0ae467d0c826ee8e0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.249597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9b616-f0b4-4a67-b5b0-72a6bca233a3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a30b5be0-375d-49b5-a3f4-be6223d6cf5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:55.249705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:55.250677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:55.258784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 88/172] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:55.259454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 84/172] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/users/1e28b9c83c4247b0ae467d0c826ee8e0 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:55.259532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b5be0-375d-49b5-a3f4-be6223d6cf5e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.259609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9b616-f0b4-4a67-b5b0-72a6bca233a3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.301962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.348609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b5be0-375d-49b5-a3f4-be6223d6cf5e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.352556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9b616-f0b4-4a67-b5b0-72a6bca233a3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.382903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8654ac-331a-4fea-b27f-5d1abd664d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.383496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8654ac-331a-4fea-b27f-5d1abd664d71 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.383768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8654ac-331a-4fea-b27f-5d1abd664d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.384034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8654ac-331a-4fea-b27f-5d1abd664d71 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.384667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8654ac-331a-4fea-b27f-5d1abd664d71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.386111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8654ac-331a-4fea-b27f-5d1abd664d71 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.386417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a56d3-e56a-48bf-b858-40b58d59c8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.387033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a56d3-e56a-48bf-b858-40b58d59c8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.387307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a56d3-e56a-48bf-b858-40b58d59c8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.387576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a56d3-e56a-48bf-b858-40b58d59c8f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.397780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a56d3-e56a-48bf-b858-40b58d59c8f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-da9a56d3-e56a-48bf-b858-40b58d59c8f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:55.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:11:55.404583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 85/173] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:55.407324 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 89/174] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:11:55 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:55.418588 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8654ac-331a-4fea-b27f-5d1abd664d71 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.418685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a56d3-e56a-48bf-b858-40b58d59c8f4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.418798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.524545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8654ac-331a-4fea-b27f-5d1abd664d71 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.525532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a56d3-e56a-48bf-b858-40b58d59c8f4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.537688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.563379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.564321 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.564733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.565244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.565928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:55.566231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.566571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256edf83-2255-47e3-aa6f-46068f5aae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.567212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256edf83-2255-47e3-aa6f-46068f5aae66 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.567484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256edf83-2255-47e3-aa6f-46068f5aae66 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.567675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.567770 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256edf83-2255-47e3-aa6f-46068f5aae66 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.569045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256edf83-2255-47e3-aa6f-46068f5aae66 None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.571209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256edf83-2255-47e3-aa6f-46068f5aae66 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.573090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 86/175] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:55.573683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256edf83-2255-47e3-aa6f-46068f5aae66 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.584341 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 90/176] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:55.585181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.657213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.691872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256edf83-2255-47e3-aa6f-46068f5aae66 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.704391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c9258-709e-4d2d-bbf9-62c4fe4e8349 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.734545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.735762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.736393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.737186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.738442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:55.738967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.741059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.742881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9c00b-c61a-4cdc-8ae9-e3741891eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.743549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9c00b-c61a-4cdc-8ae9-e3741891eeaa None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.743822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9c00b-c61a-4cdc-8ae9-e3741891eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.744091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9c00b-c61a-4cdc-8ae9-e3741891eeaa None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185/users/1e28b9c83c4247b0ae467d0c826ee8e0/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.745979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9c00b-c61a-4cdc-8ae9-e3741891eeaa None admin] RBAC: Authorizing `identity:create_grant(project_id=102d784de27145ad9641bd35a5ea7185, user_id=1e28b9c83c4247b0ae467d0c826ee8e0, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.747452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9c00b-c61a-4cdc-8ae9-e3741891eeaa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.754815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 87/177] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:55.755242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.765689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 91/178] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:55 2022] PUT /identity/v3/projects/102d784de27145ad9641bd35a5ea7185/users/1e28b9c83c4247b0ae467d0c826ee8e0/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:11:55.770432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9c00b-c61a-4cdc-8ae9-e3741891eeaa None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.813753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.897991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe0dcd-ea2b-4aad-8d78-24c816d5c8d3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.901059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9c00b-c61a-4cdc-8ae9-e3741891eeaa None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:55.937414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:55.945978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8819e-553a-4048-bf2b-3fa84915c3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.947114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8819e-553a-4048-bf2b-3fa84915c3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.947561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8819e-553a-4048-bf2b-3fa84915c3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.948012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8819e-553a-4048-bf2b-3fa84915c3f3 None admin] PATH_INFO: `/v3/users/80ba9dacb6bd48e39aeca10439ce8d25` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.949518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8819e-553a-4048-bf2b-3fa84915c3f3 None admin] RBAC: Authorizing `identity:get_user(user_id=80ba9dacb6bd48e39aeca10439ce8d25)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.951275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8819e-553a-4048-bf2b-3fa84915c3f3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.953187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 88/179] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/users/80ba9dacb6bd48e39aeca10439ce8d25 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:55.960739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8819e-553a-4048-bf2b-3fa84915c3f3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:55.961981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f3c57-aa8e-44f0-8d23-4b6822544ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:55.964135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f3c57-aa8e-44f0-8d23-4b6822544ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:55.964135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f3c57-aa8e-44f0-8d23-4b6822544ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:55.964135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f3c57-aa8e-44f0-8d23-4b6822544ad5 None admin] PATH_INFO: `/v3/users/80ba9dacb6bd48e39aeca10439ce8d25` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:55.964388 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f3c57-aa8e-44f0-8d23-4b6822544ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=80ba9dacb6bd48e39aeca10439ce8d25)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:55.965755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f3c57-aa8e-44f0-8d23-4b6822544ad5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:55.967839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 92/180] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/users/80ba9dacb6bd48e39aeca10439ce8d25 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:55.973276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f3c57-aa8e-44f0-8d23-4b6822544ad5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:56.086265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8819e-553a-4048-bf2b-3fa84915c3f3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:56.095819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:56.117170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f3c57-aa8e-44f0-8d23-4b6822544ad5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:56.154955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc4a41-c3ce-4a36-9eda-4951ee9f4004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:56.155932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc4a41-c3ce-4a36-9eda-4951ee9f4004 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:56.156398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc4a41-c3ce-4a36-9eda-4951ee9f4004 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:56.156853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc4a41-c3ce-4a36-9eda-4951ee9f4004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:56.163028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6d6d7-35b0-4a44-94c7-c9fd0ca3d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:56.163652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6d6d7-35b0-4a44-94c7-c9fd0ca3d633 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:56.163933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6d6d7-35b0-4a44-94c7-c9fd0ca3d633 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:56.164198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6d6d7-35b0-4a44-94c7-c9fd0ca3d633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:56.229037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:56.241472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc4a41-c3ce-4a36-9eda-4951ee9f4004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:56.243923 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc4a41-c3ce-4a36-9eda-4951ee9f4004 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:56.288555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6d6d7-35b0-4a44-94c7-c9fd0ca3d633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:56.290171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6d6d7-35b0-4a44-94c7-c9fd0ca3d633 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:56.404221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 93/181] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 10:11:56.410159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc4a41-c3ce-4a36-9eda-4951ee9f4004 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:56.427822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 89/182] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:11:56.436584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6d6d7-35b0-4a44-94c7-c9fd0ca3d633 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:56.453304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:56.554807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:56.564495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6d6d7-35b0-4a44-94c7-c9fd0ca3d633 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:56.566574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc4a41-c3ce-4a36-9eda-4951ee9f4004 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:56.630687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b44f5-eae2-4636-9767-707d2c021d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:56.631265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b44f5-eae2-4636-9767-707d2c021d75 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:56.631536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b44f5-eae2-4636-9767-707d2c021d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:56.631803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b44f5-eae2-4636-9767-707d2c021d75 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:56.633272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b44f5-eae2-4636-9767-707d2c021d75 None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:56.634960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b44f5-eae2-4636-9767-707d2c021d75 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:56.640818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcf8b0-be7e-45ae-bfd4-0922ceb2a3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:56.641460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcf8b0-be7e-45ae-bfd4-0922ceb2a3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:56.641733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcf8b0-be7e-45ae-bfd4-0922ceb2a3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:56.642002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcf8b0-be7e-45ae-bfd4-0922ceb2a3f9 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:56.647186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 90/183] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:56 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:56.648203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b44f5-eae2-4636-9767-707d2c021d75 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:56.649266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcf8b0-be7e-45ae-bfd4-0922ceb2a3f9 None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:56.653477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcf8b0-be7e-45ae-bfd4-0922ceb2a3f9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:56.655282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 94/184] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:56 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:56.663627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcf8b0-be7e-45ae-bfd4-0922ceb2a3f9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:56.804325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:56.857254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b44f5-eae2-4636-9767-707d2c021d75 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:56.877312 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcf8b0-be7e-45ae-bfd4-0922ceb2a3f9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:56.925413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f98cf-9288-46e8-aeaf-0786ee866aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:56.926220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f98cf-9288-46e8-aeaf-0786ee866aaf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:56.926654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f98cf-9288-46e8-aeaf-0786ee866aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:56.927414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f98cf-9288-46e8-aeaf-0786ee866aaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:56.928478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f98cf-9288-46e8-aeaf-0786ee866aaf None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=102d784de27145ad9641bd35a5ea7185, user.id=80ba9dacb6bd48e39aeca10439ce8d25) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:56.928928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f98cf-9288-46e8-aeaf-0786ee866aaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:56.930518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f98cf-9288-46e8-aeaf-0786ee866aaf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:56.942455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 91/185] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:11:56 2022] GET /identity/v3/role_assignments?user.id=80ba9dacb6bd48e39aeca10439ce8d25&scope.project.id=102d784de27145ad9641bd35a5ea7185&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:56.947118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf70ce-ca04-4ef4-b1bd-3944f9e43c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:56.947757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf70ce-ca04-4ef4-b1bd-3944f9e43c1a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:56.948037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf70ce-ca04-4ef4-b1bd-3944f9e43c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:56.948306 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf70ce-ca04-4ef4-b1bd-3944f9e43c1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:56.949055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf70ce-ca04-4ef4-b1bd-3944f9e43c1a None admin] RBAC: Adding query filter params (role.id=eaef1c14dd23498185dccc05b242ff6a, scope.project.id=102d784de27145ad9641bd35a5ea7185, user.id=80ba9dacb6bd48e39aeca10439ce8d25) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:56.949362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf70ce-ca04-4ef4-b1bd-3944f9e43c1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:56.950875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf70ce-ca04-4ef4-b1bd-3944f9e43c1a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:56.968747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 95/186] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:11:56 2022] GET /identity/v3/role_assignments?user.id=80ba9dacb6bd48e39aeca10439ce8d25&scope.project.id=102d784de27145ad9641bd35a5ea7185&role.id=eaef1c14dd23498185dccc05b242ff6a => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:57.990796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf475c-45b4-4640-9615-ff3664287aa6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:57.991342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf475c-45b4-4640-9615-ff3664287aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:57.991342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf475c-45b4-4640-9615-ff3664287aa6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:57.992708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 92/187] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:57.999110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ce235-f1ef-49da-8968-8fdfb6fd6890 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:57.999625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ce235-f1ef-49da-8968-8fdfb6fd6890 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:57.999625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ce235-f1ef-49da-8968-8fdfb6fd6890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.018266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8b6ce235-f1ef-49da-8968-8fdfb6fd6890 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:58.019463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8b6ce235-f1ef-49da-8968-8fdfb6fd6890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:58.037517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:58.096049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 96/188] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:58.110413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258ff1-3176-4740-9923-42dc74c69413 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.110693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258ff1-3176-4740-9923-42dc74c69413 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.110962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258ff1-3176-4740-9923-42dc74c69413 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.112311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 93/189] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:11:58.118155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa528f6b-33dd-4fb2-b75e-c8a89755e3ec None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.118475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa528f6b-33dd-4fb2-b75e-c8a89755e3ec None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.118746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa528f6b-33dd-4fb2-b75e-c8a89755e3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.136901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fa528f6b-33dd-4fb2-b75e-c8a89755e3ec None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:58.138049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fa528f6b-33dd-4fb2-b75e-c8a89755e3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:58.169712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22768e85-09cc-4733-b25e-4584ba22319e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.169994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22768e85-09cc-4733-b25e-4584ba22319e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.170263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22768e85-09cc-4733-b25e-4584ba22319e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.171745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 94/190] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:11:58.176468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbdbb6-5f28-48f0-829e-f6d5e38a41bf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.176747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbdbb6-5f28-48f0-829e-f6d5e38a41bf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.177013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbdbb6-5f28-48f0-829e-f6d5e38a41bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.197185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f8cbdbb6-5f28-48f0-829e-f6d5e38a41bf None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:11:58.198370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f8cbdbb6-5f28-48f0-829e-f6d5e38a41bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:11:58.214373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 97/191] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:11:58.270372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 95/192] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:11:58.375282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa528f6b-33dd-4fb2-b75e-c8a89755e3ec None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:58.460945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:58.478678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa528f6b-33dd-4fb2-b75e-c8a89755e3ec None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:58.487978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbdbb6-5f28-48f0-829e-f6d5e38a41bf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:58.515724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dbfcb-3c26-4f93-bec0-c1a43aab3f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:58.516803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dbfcb-3c26-4f93-bec0-c1a43aab3f58 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.517266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dbfcb-3c26-4f93-bec0-c1a43aab3f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.517540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dbfcb-3c26-4f93-bec0-c1a43aab3f58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.529968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dbfcb-3c26-4f93-bec0-c1a43aab3f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-533dbfcb-3c26-4f93-bec0-c1a43aab3f58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:58.535114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:58.536218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 98/193] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:58.537975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dbfcb-3c26-4f93-bec0-c1a43aab3f58 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:58.563243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:58.594478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbdbb6-5f28-48f0-829e-f6d5e38a41bf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:58.631724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a38492-1c78-4c98-8fc2-97dd12952d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:58.632358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38492-1c78-4c98-8fc2-97dd12952d15 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.632635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38492-1c78-4c98-8fc2-97dd12952d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.632903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a38492-1c78-4c98-8fc2-97dd12952d15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.634021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a38492-1c78-4c98-8fc2-97dd12952d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:58.635492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a38492-1c78-4c98-8fc2-97dd12952d15 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:58.637075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 96/194] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:11:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:58.637829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a38492-1c78-4c98-8fc2-97dd12952d15 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:58.642735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dbfcb-3c26-4f93-bec0-c1a43aab3f58 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:58.677030 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60e10e-2e70-47ad-89e1-2167b829cb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:58.678850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e10e-2e70-47ad-89e1-2167b829cb91 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.679124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e10e-2e70-47ad-89e1-2167b829cb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.679390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e10e-2e70-47ad-89e1-2167b829cb91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.689501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60e10e-2e70-47ad-89e1-2167b829cb91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ef60e10e-2e70-47ad-89e1-2167b829cb91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:11:58.695544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:11:58.696692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:11:58.697621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:11:58.701642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 99/195] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:11:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:11:58.702516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60e10e-2e70-47ad-89e1-2167b829cb91 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:58.706673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:58.744903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a38492-1c78-4c98-8fc2-97dd12952d15 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:58.777941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:58.778578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.778851 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.779117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.779784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:58.780151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:58.781451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:58.786351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:58.790549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 97/196] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:58.791340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:58.807452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60e10e-2e70-47ad-89e1-2167b829cb91 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:58.841564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79032a1-b433-4859-9b19-419bb89ea585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:58.842168 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79032a1-b433-4859-9b19-419bb89ea585 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.842479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79032a1-b433-4859-9b19-419bb89ea585 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.842750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79032a1-b433-4859-9b19-419bb89ea585 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.843396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79032a1-b433-4859-9b19-419bb89ea585 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:58.844932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79032a1-b433-4859-9b19-419bb89ea585 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:58.865758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 100/197] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:11:58 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:11:58.866606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79032a1-b433-4859-9b19-419bb89ea585 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:58.897870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979f5ce-1173-4cde-a02e-e7a8419e2dfc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:58.934861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:58.946383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:58.947190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:58.947471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:58.948599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:58.949313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:58.949618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:58.951164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:58.963574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 98/198] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:11:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:58.969000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:58.982255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79032a1-b433-4859-9b19-419bb89ea585 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:59.018775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c5625-cafc-4ee0-8414-3ca8c9bd5ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:59.019452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5625-cafc-4ee0-8414-3ca8c9bd5ecc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:59.019676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5625-cafc-4ee0-8414-3ca8c9bd5ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:59.019941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5625-cafc-4ee0-8414-3ca8c9bd5ecc None admin] PATH_INFO: `/v3/users/b1592a66520e4f2abf0e30b36bc77cc2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:59.034174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c5625-cafc-4ee0-8414-3ca8c9bd5ecc None admin] RBAC: Authorizing `identity:get_user(user_id=b1592a66520e4f2abf0e30b36bc77cc2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:59.035575 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c5625-cafc-4ee0-8414-3ca8c9bd5ecc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:59.037940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 101/199] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:58 2022] GET /identity/v3/users/b1592a66520e4f2abf0e30b36bc77cc2 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:59.043335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c5625-cafc-4ee0-8414-3ca8c9bd5ecc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:59.045270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:59.098785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e812a0b-c529-4613-bfbe-be9a823b3ff9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:59.141328 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d9fa6-9247-414c-adb6-68c7b3c0e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:59.141967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d9fa6-9247-414c-adb6-68c7b3c0e519 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:59.142238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d9fa6-9247-414c-adb6-68c7b3c0e519 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:59.142593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d9fa6-9247-414c-adb6-68c7b3c0e519 None admin] PATH_INFO: `/v3/users/1e28b9c83c4247b0ae467d0c826ee8e0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:59.144547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d9fa6-9247-414c-adb6-68c7b3c0e519 None admin] RBAC: Authorizing `identity:get_user(user_id=1e28b9c83c4247b0ae467d0c826ee8e0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:59.146998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d9fa6-9247-414c-adb6-68c7b3c0e519 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:59.148804 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 99/200] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:11:58 2022] GET /identity/v3/users/1e28b9c83c4247b0ae467d0c826ee8e0 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:59.154573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d9fa6-9247-414c-adb6-68c7b3c0e519 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:59.168014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c5625-cafc-4ee0-8414-3ca8c9bd5ecc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:59.188055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:59.207766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43942f-e017-45f7-9044-d66f4b644f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:59.209356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43942f-e017-45f7-9044-d66f4b644f66 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:59.209467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43942f-e017-45f7-9044-d66f4b644f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:59.209695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43942f-e017-45f7-9044-d66f4b644f66 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:59.210917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43942f-e017-45f7-9044-d66f4b644f66 None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:59.212339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43942f-e017-45f7-9044-d66f4b644f66 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:59.214274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 102/201] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:59 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:11:59.220128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43942f-e017-45f7-9044-d66f4b644f66 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:59.281604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d9fa6-9247-414c-adb6-68c7b3c0e519 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:59.316171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:59.322795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4053b8d-dfa7-4ea7-a19a-ee7a119f4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:59.323399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4053b8d-dfa7-4ea7-a19a-ee7a119f4395 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:59.323672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4053b8d-dfa7-4ea7-a19a-ee7a119f4395 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:59.324199 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4053b8d-dfa7-4ea7-a19a-ee7a119f4395 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:59.347893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43942f-e017-45f7-9044-d66f4b644f66 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:59.398628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda36430-e8b9-4f75-94dc-5fe9163cd76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:59.399567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda36430-e8b9-4f75-94dc-5fe9163cd76d None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:59.399845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda36430-e8b9-4f75-94dc-5fe9163cd76d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:59.400119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda36430-e8b9-4f75-94dc-5fe9163cd76d None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185/users/b1592a66520e4f2abf0e30b36bc77cc2/roles/c1cacdb19cc543428839254e4ecf77a2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:59.402129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda36430-e8b9-4f75-94dc-5fe9163cd76d None admin] RBAC: Authorizing `identity:create_grant(project_id=102d784de27145ad9641bd35a5ea7185, user_id=b1592a66520e4f2abf0e30b36bc77cc2, role_id=c1cacdb19cc543428839254e4ecf77a2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:59.404475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda36430-e8b9-4f75-94dc-5fe9163cd76d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:59.404677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4053b8d-dfa7-4ea7-a19a-ee7a119f4395 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:59.406165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4053b8d-dfa7-4ea7-a19a-ee7a119f4395 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:59.419081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 103/202] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:11:59 2022] PUT /identity/v3/projects/102d784de27145ad9641bd35a5ea7185/users/b1592a66520e4f2abf0e30b36bc77cc2/roles/c1cacdb19cc543428839254e4ecf77a2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:11:59.526813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 100/203] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 10:11:59.532807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda36430-e8b9-4f75-94dc-5fe9163cd76d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:59.585766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:59.644336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda36430-e8b9-4f75-94dc-5fe9163cd76d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:59.679346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b5dc8-0a5a-41ab-ba98-2ef8d730db0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:59.680038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b5dc8-0a5a-41ab-ba98-2ef8d730db0b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:59.680319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b5dc8-0a5a-41ab-ba98-2ef8d730db0b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:59.680587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b5dc8-0a5a-41ab-ba98-2ef8d730db0b None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:59.681712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b5dc8-0a5a-41ab-ba98-2ef8d730db0b None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:59.683175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b5dc8-0a5a-41ab-ba98-2ef8d730db0b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:59.689782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 104/204] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:11:59 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:11:59.691162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4053b8d-dfa7-4ea7-a19a-ee7a119f4395 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:11:59.728094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:11:59.797589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4053b8d-dfa7-4ea7-a19a-ee7a119f4395 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:11:59.833645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534052-0cce-4ec5-99c1-a2266b8d92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:11:59.834309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534052-0cce-4ec5-99c1-a2266b8d92fa None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:11:59.834648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534052-0cce-4ec5-99c1-a2266b8d92fa None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:11:59.834923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534052-0cce-4ec5-99c1-a2266b8d92fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:11:59.835652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78534052-0cce-4ec5-99c1-a2266b8d92fa None admin] RBAC: Adding query filter params (role.id=cbe9824e14834e26a81ef6a408bd1bd4, scope.project.id=102d784de27145ad9641bd35a5ea7185, user.id=1e28b9c83c4247b0ae467d0c826ee8e0) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:11:59.836026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78534052-0cce-4ec5-99c1-a2266b8d92fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:11:59.837387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78534052-0cce-4ec5-99c1-a2266b8d92fa None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:11:59.850560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 101/205] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:11:59 2022] GET /identity/v3/role_assignments?user.id=1e28b9c83c4247b0ae467d0c826ee8e0&scope.project.id=102d784de27145ad9641bd35a5ea7185&role.id=cbe9824e14834e26a81ef6a408bd1bd4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:01.218154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789ee96-1f2d-4c92-aa33-edeab4921479 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.218704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789ee96-1f2d-4c92-aa33-edeab4921479 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.218704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789ee96-1f2d-4c92-aa33-edeab4921479 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.220240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 105/206] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:01.225736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6101f6-b8cb-4294-8532-ce6cdebbbc5b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.226133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6101f6-b8cb-4294-8532-ce6cdebbbc5b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.226309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6101f6-b8cb-4294-8532-ce6cdebbbc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.244937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ed6101f6-b8cb-4294-8532-ce6cdebbbc5b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:01.246121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ed6101f6-b8cb-4294-8532-ce6cdebbbc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:01.281965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564cda-32d0-442e-96e7-fe1c800a5a27 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.282268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564cda-32d0-442e-96e7-fe1c800a5a27 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.282573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564cda-32d0-442e-96e7-fe1c800a5a27 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.284097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 106/207] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:01.289615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46d909-4910-4339-adb1-d69f5e4a653c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.290548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46d909-4910-4339-adb1-d69f5e4a653c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.290824 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46d909-4910-4339-adb1-d69f5e4a653c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.309836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5e46d909-4910-4339-adb1-d69f5e4a653c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:01.311166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5e46d909-4910-4339-adb1-d69f5e4a653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:01.316498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 102/208] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:01.383379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 107/209] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:01.589360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6101f6-b8cb-4294-8532-ce6cdebbbc5b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:01.615847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:01.617804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704280-7e3b-48c0-9d1a-f419fe228138 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.618131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704280-7e3b-48c0-9d1a-f419fe228138 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.618363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704280-7e3b-48c0-9d1a-f419fe228138 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.619719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 108/210] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:01.624581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40d4c-649e-4b49-8094-394370cd457b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.624860 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40d4c-649e-4b49-8094-394370cd457b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.625131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40d4c-649e-4b49-8094-394370cd457b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.644102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-22b40d4c-649e-4b49-8094-394370cd457b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:01.645243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-22b40d4c-649e-4b49-8094-394370cd457b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:01.684743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:01.694498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6101f6-b8cb-4294-8532-ce6cdebbbc5b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:01.711781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 109/211] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:01.712813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40d4c-649e-4b49-8094-394370cd457b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:01.732026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dc3f8-a951-4c95-8193-43c2f42cae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:01.732560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dc3f8-a951-4c95-8193-43c2f42cae1c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.732835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dc3f8-a951-4c95-8193-43c2f42cae1c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.733102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dc3f8-a951-4c95-8193-43c2f42cae1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.744442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98dc3f8-a951-4c95-8193-43c2f42cae1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c98dc3f8-a951-4c95-8193-43c2f42cae1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:01.749213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:01.754573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:01.756311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 103/212] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:01.758049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dc3f8-a951-4c95-8193-43c2f42cae1c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:01.826659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40d4c-649e-4b49-8094-394370cd457b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:01.845686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:01.861256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8618639-12f7-456c-9525-eac3c786f19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:01.861881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8618639-12f7-456c-9525-eac3c786f19b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.862158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8618639-12f7-456c-9525-eac3c786f19b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.862456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8618639-12f7-456c-9525-eac3c786f19b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.863549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8618639-12f7-456c-9525-eac3c786f19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:01.866241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dc3f8-a951-4c95-8193-43c2f42cae1c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:01.866601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8618639-12f7-456c-9525-eac3c786f19b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:01.868194 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 110/213] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:01.873521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8618639-12f7-456c-9525-eac3c786f19b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:01.908100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:01.908571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:01.908843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:01.909109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:01.909999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:01.910106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:01.911497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:01.918518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:01.919683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 104/214] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:01.925234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:01.983200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8618639-12f7-456c-9525-eac3c786f19b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:02.017385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f054-091f-4c8a-90f8-752f57c92172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:02.017983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f054-091f-4c8a-90f8-752f57c92172 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:02.018257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f054-091f-4c8a-90f8-752f57c92172 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:02.018576 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f054-091f-4c8a-90f8-752f57c92172 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:02.019223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7f054-091f-4c8a-90f8-752f57c92172 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:02.020671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7f054-091f-4c8a-90f8-752f57c92172 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:02.026163 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472eaa69-b612-4e6f-88af-5661bad6bb2c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:02.042669 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 111/215] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:12:01 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:12:02.047262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f054-091f-4c8a-90f8-752f57c92172 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:02.062357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d2d96-395e-4480-9e18-734aee929321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:02.063027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d2d96-395e-4480-9e18-734aee929321 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:02.063309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d2d96-395e-4480-9e18-734aee929321 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:02.063582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d2d96-395e-4480-9e18-734aee929321 None admin] PATH_INFO: `/v3/users/f217bd5b4a214ba29d98b9d4f5cd3c48` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:02.086386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d2d96-395e-4480-9e18-734aee929321 None admin] RBAC: Authorizing `identity:get_user(user_id=f217bd5b4a214ba29d98b9d4f5cd3c48)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:02.088057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d2d96-395e-4480-9e18-734aee929321 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:02.090310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 105/216] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:12:01 2022] GET /identity/v3/users/f217bd5b4a214ba29d98b9d4f5cd3c48 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:02.100224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d2d96-395e-4480-9e18-734aee929321 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:02.106623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:02.164571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:02.169082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f054-091f-4c8a-90f8-752f57c92172 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:02.207661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709922cc-51cf-4a59-b091-85733fdedcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:02.208400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709922cc-51cf-4a59-b091-85733fdedcb5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:02.209327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709922cc-51cf-4a59-b091-85733fdedcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:02.209602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709922cc-51cf-4a59-b091-85733fdedcb5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:02.221837 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709922cc-51cf-4a59-b091-85733fdedcb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-709922cc-51cf-4a59-b091-85733fdedcb5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:02.228149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:02.229046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:02.229843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 112/217] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:02.233309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d2d96-395e-4480-9e18-734aee929321 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:02.234988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709922cc-51cf-4a59-b091-85733fdedcb5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:02.273868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:02.274872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:02.275148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:02.275412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:02.276140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] RBAC: Adding query filter params (role.id=cbe9824e14834e26a81ef6a408bd1bd4, user.id=f217bd5b4a214ba29d98b9d4f5cd3c48, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:02.276442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:02.277868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:02.287469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 106/218] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Oct 17 10:12:02 2022] GET /identity/v3/role_assignments?user.id=f217bd5b4a214ba29d98b9d4f5cd3c48&scope.system=all&role.id=cbe9824e14834e26a81ef6a408bd1bd4 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:02.354393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709922cc-51cf-4a59-b091-85733fdedcb5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:02.397892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:02.398551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:02.398828 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:02.399096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:02.399768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:02.400069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:02.401465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:02.413300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 113/219] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:02.419065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:02.499912 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:02.534494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e986-e0cb-49ab-9f93-605cc57e47e5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:02.572655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253fb5d-51b0-47b2-8ea4-7c3514884f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:02.573292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253fb5d-51b0-47b2-8ea4-7c3514884f60 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:02.573566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253fb5d-51b0-47b2-8ea4-7c3514884f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:02.573837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253fb5d-51b0-47b2-8ea4-7c3514884f60 None admin] PATH_INFO: `/v3/users/b1592a66520e4f2abf0e30b36bc77cc2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:02.575005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253fb5d-51b0-47b2-8ea4-7c3514884f60 None admin] RBAC: Authorizing `identity:get_user(user_id=b1592a66520e4f2abf0e30b36bc77cc2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:02.576509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253fb5d-51b0-47b2-8ea4-7c3514884f60 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:02.578491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 107/220] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:12:02 2022] GET /identity/v3/users/b1592a66520e4f2abf0e30b36bc77cc2 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:02.584392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:02.608645 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:02.694183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039673f2-0746-4113-9e3d-c028e950e221 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:02.730887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083e537-f8dc-4575-bd1b-4ff3208c6d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:02.731862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083e537-f8dc-4575-bd1b-4ff3208c6d22 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:02.732331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083e537-f8dc-4575-bd1b-4ff3208c6d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:02.732792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083e537-f8dc-4575-bd1b-4ff3208c6d22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:02.806344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083e537-f8dc-4575-bd1b-4ff3208c6d22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:02.808143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083e537-f8dc-4575-bd1b-4ff3208c6d22 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:02.829173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:02.912481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 114/221] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 10:12:02.918024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253fb5d-51b0-47b2-8ea4-7c3514884f60 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:03.021924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253fb5d-51b0-47b2-8ea4-7c3514884f60 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:03.057309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27490f18-1867-4b3b-bd27-104418e3be88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:03.057938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27490f18-1867-4b3b-bd27-104418e3be88 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:03.058208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27490f18-1867-4b3b-bd27-104418e3be88 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:03.058522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27490f18-1867-4b3b-bd27-104418e3be88 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:03.059647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27490f18-1867-4b3b-bd27-104418e3be88 None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:03.061060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27490f18-1867-4b3b-bd27-104418e3be88 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:03.063071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 108/222] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:12:02 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:03.069109 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083e537-f8dc-4575-bd1b-4ff3208c6d22 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:03.173647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083e537-f8dc-4575-bd1b-4ff3208c6d22 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:03.197999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:03.215926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5c1d6-e43e-42cb-8b6b-cb5b5f3375ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:03.216960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c1d6-e43e-42cb-8b6b-cb5b5f3375ae None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:03.217441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c1d6-e43e-42cb-8b6b-cb5b5f3375ae None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:03.217940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c1d6-e43e-42cb-8b6b-cb5b5f3375ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:03.218975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5c1d6-e43e-42cb-8b6b-cb5b5f3375ae None admin] RBAC: Adding query filter params (role.id=c1cacdb19cc543428839254e4ecf77a2, scope.project.id=102d784de27145ad9641bd35a5ea7185, user.id=b1592a66520e4f2abf0e30b36bc77cc2) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:03.219507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5c1d6-e43e-42cb-8b6b-cb5b5f3375ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:03.221218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5c1d6-e43e-42cb-8b6b-cb5b5f3375ae None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:03.228875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 115/223] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:03 2022] GET /identity/v3/role_assignments?user.id=b1592a66520e4f2abf0e30b36bc77cc2&scope.project.id=102d784de27145ad9641bd35a5ea7185&role.id=c1cacdb19cc543428839254e4ecf77a2 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:04.433316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feebbee-f55a-40c5-9448-d619f3f8dd16 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:04.433809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feebbee-f55a-40c5-9448-d619f3f8dd16 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:04.433809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feebbee-f55a-40c5-9448-d619f3f8dd16 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:04.435157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 109/224] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:04.440201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3207a1-6e0e-446a-9c94-94a624a0dec5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:04.440623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3207a1-6e0e-446a-9c94-94a624a0dec5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:04.440807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3207a1-6e0e-446a-9c94-94a624a0dec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:04.460313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ee3207a1-6e0e-446a-9c94-94a624a0dec5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:04.461480 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ee3207a1-6e0e-446a-9c94-94a624a0dec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:04.502151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf79a1-7cc7-41fd-8c6c-7800b0004f45 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:04.502587 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf79a1-7cc7-41fd-8c6c-7800b0004f45 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:04.502776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf79a1-7cc7-41fd-8c6c-7800b0004f45 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:04.504106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 110/225] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:04.509294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315664e3-3ba2-45cd-b114-78d98a72fd06 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:04.509569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315664e3-3ba2-45cd-b114-78d98a72fd06 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:04.509842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315664e3-3ba2-45cd-b114-78d98a72fd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:04.516835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:04.528659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-315664e3-3ba2-45cd-b114-78d98a72fd06 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:04.529823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-315664e3-3ba2-45cd-b114-78d98a72fd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:04.532060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 116/226] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:04.533300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818715a2-e66a-4f4d-a7d2-e5d5a8f98c65 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:04.533622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818715a2-e66a-4f4d-a7d2-e5d5a8f98c65 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:04.534513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818715a2-e66a-4f4d-a7d2-e5d5a8f98c65 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:04.535224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 117/227] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:04.540388 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e637896-75e0-4342-ad36-674150f5785c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:04.540667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e637896-75e0-4342-ad36-674150f5785c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:04.540939 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e637896-75e0-4342-ad36-674150f5785c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:04.559519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5e637896-75e0-4342-ad36-674150f5785c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:04.560756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5e637896-75e0-4342-ad36-674150f5785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:04.606979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 111/228] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:04.630494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 118/229] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:04.804138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315664e3-3ba2-45cd-b114-78d98a72fd06 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:04.886658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e637896-75e0-4342-ad36-674150f5785c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:04.906924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315664e3-3ba2-45cd-b114-78d98a72fd06 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:04.940808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:04.948622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7d667-212f-476f-a5e3-7059d37a99db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:04.949441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7d667-212f-476f-a5e3-7059d37a99db None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:04.950288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7d667-212f-476f-a5e3-7059d37a99db None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:04.950740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7d667-212f-476f-a5e3-7059d37a99db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:04.952983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:04.963693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7d667-212f-476f-a5e3-7059d37a99db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b6a7d667-212f-476f-a5e3-7059d37a99db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:04.969707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:04.970836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:12:04.971859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:04.974983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 112/230] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:04.976026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7d667-212f-476f-a5e3-7059d37a99db None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:05.003876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e637896-75e0-4342-ad36-674150f5785c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.050938 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de844e9-64a9-4e54-9542-1bed45a589e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.051467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de844e9-64a9-4e54-9542-1bed45a589e8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.051739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de844e9-64a9-4e54-9542-1bed45a589e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.052647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de844e9-64a9-4e54-9542-1bed45a589e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.063001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de844e9-64a9-4e54-9542-1bed45a589e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9de844e9-64a9-4e54-9542-1bed45a589e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:05.067899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:05.069286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 119/231] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:05.070092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de844e9-64a9-4e54-9542-1bed45a589e8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:05.098547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7d667-212f-476f-a5e3-7059d37a99db None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.133226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f190f4a-2069-46ce-9473-58f1df66b2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.133849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f190f4a-2069-46ce-9473-58f1df66b2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.134119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f190f4a-2069-46ce-9473-58f1df66b2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.134419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f190f4a-2069-46ce-9473-58f1df66b2f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.135504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f190f4a-2069-46ce-9473-58f1df66b2f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:05.136926 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f190f4a-2069-46ce-9473-58f1df66b2f3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:05.138512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 113/232] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:05.139260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f190f4a-2069-46ce-9473-58f1df66b2f3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:05.188717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de844e9-64a9-4e54-9542-1bed45a589e8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.191195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:05.212134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:05.229411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.230017 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.230290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.230598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.231261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:05.231562 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:05.232945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:05.242560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 120/233] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:05.243357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:05.258636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f190f4a-2069-46ce-9473-58f1df66b2f3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.294259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.295039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.295311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.295577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.296234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:05.296528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:05.297878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:05.306564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 114/234] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:05.307258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:05.353235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112b664-3440-4bc1-8664-08c2ae148b03 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.387096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c6fc7-01ef-4746-9c5b-2a3ce0080070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.387702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c6fc7-01ef-4746-9c5b-2a3ce0080070 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.387980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c6fc7-01ef-4746-9c5b-2a3ce0080070 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.388253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c6fc7-01ef-4746-9c5b-2a3ce0080070 None admin] PATH_INFO: `/v3/groups` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.388878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c6fc7-01ef-4746-9c5b-2a3ce0080070 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:05.390351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c6fc7-01ef-4746-9c5b-2a3ce0080070 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:05.405936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 121/235] 10.222.0.16 () {60 vars in 1134 bytes} [Mon Oct 17 10:12:05 2022] POST /identity/v3/groups => generated 229 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:05.410546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ffc03-bebe-4d17-8305-fc448527cdae None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.416119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c6fc7-01ef-4746-9c5b-2a3ce0080070 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:05.435290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:05.449233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:05.449794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe80aa-7fc9-4d9e-b166-512c2eb3b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.450559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe80aa-7fc9-4d9e-b166-512c2eb3b5aa None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.450835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe80aa-7fc9-4d9e-b166-512c2eb3b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.451108 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe80aa-7fc9-4d9e-b166-512c2eb3b5aa None admin] PATH_INFO: `/v3/users/07abfeec43dd4275a229b0999df294d9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.471292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe80aa-7fc9-4d9e-b166-512c2eb3b5aa None admin] RBAC: Authorizing `identity:get_user(user_id=07abfeec43dd4275a229b0999df294d9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:05.473027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe80aa-7fc9-4d9e-b166-512c2eb3b5aa None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:05.479634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 115/236] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:12:05 2022] GET /identity/v3/users/07abfeec43dd4275a229b0999df294d9 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:05.480860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe80aa-7fc9-4d9e-b166-512c2eb3b5aa None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:05.545946 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c6fc7-01ef-4746-9c5b-2a3ce0080070 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.587815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9a89d-2221-484d-834e-96f4652d59cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.588456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9a89d-2221-484d-834e-96f4652d59cb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.588738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9a89d-2221-484d-834e-96f4652d59cb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.589013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9a89d-2221-484d-834e-96f4652d59cb None admin] PATH_INFO: `/v3/users/f217bd5b4a214ba29d98b9d4f5cd3c48` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.590156 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9a89d-2221-484d-834e-96f4652d59cb None admin] RBAC: Authorizing `identity:get_user(user_id=f217bd5b4a214ba29d98b9d4f5cd3c48)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:05.591676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9a89d-2221-484d-834e-96f4652d59cb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:05.593277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 122/237] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:12:05 2022] GET /identity/v3/users/f217bd5b4a214ba29d98b9d4f5cd3c48 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:05.596930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe80aa-7fc9-4d9e-b166-512c2eb3b5aa None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.599879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9a89d-2221-484d-834e-96f4652d59cb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:05.639317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b357e87-6621-46fc-801e-de2206259a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.639944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b357e87-6621-46fc-801e-de2206259a15 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.640218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b357e87-6621-46fc-801e-de2206259a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.641155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b357e87-6621-46fc-801e-de2206259a15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.641332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b357e87-6621-46fc-801e-de2206259a15 None admin] RBAC: Adding query filter params (role.id=c1cacdb19cc543428839254e4ecf77a2, user.id=07abfeec43dd4275a229b0999df294d9, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:05.641616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b357e87-6621-46fc-801e-de2206259a15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:05.643065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b357e87-6621-46fc-801e-de2206259a15 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:05.656929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 116/238] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Oct 17 10:12:05 2022] GET /identity/v3/role_assignments?user.id=07abfeec43dd4275a229b0999df294d9&scope.system=all&role.id=c1cacdb19cc543428839254e4ecf77a2 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:05.719665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:05.729295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9a89d-2221-484d-834e-96f4652d59cb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:05.766468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dd686-a2e7-4d9f-aadb-6357661a8071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:05.767143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd686-a2e7-4d9f-aadb-6357661a8071 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:05.767423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd686-a2e7-4d9f-aadb-6357661a8071 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:05.767697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd686-a2e7-4d9f-aadb-6357661a8071 None admin] PATH_INFO: `/v3/system/users/f217bd5b4a214ba29d98b9d4f5cd3c48/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:05.769216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dd686-a2e7-4d9f-aadb-6357661a8071 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f217bd5b4a214ba29d98b9d4f5cd3c48, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:05.770644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dd686-a2e7-4d9f-aadb-6357661a8071 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:05.781749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 123/239] 10.222.0.16 () {58 vars in 1334 bytes} [Mon Oct 17 10:12:05 2022] PUT /identity/v3/system/users/f217bd5b4a214ba29d98b9d4f5cd3c48/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:12:07.722342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ba8d6-5876-4994-8944-489b2130baf4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:07.722895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ba8d6-5876-4994-8944-489b2130baf4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:07.722895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ba8d6-5876-4994-8944-489b2130baf4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:07.724526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 117/240] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:07.731492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65125a-cbbe-47c8-9d00-72c17f4e0b44 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:07.732155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65125a-cbbe-47c8-9d00-72c17f4e0b44 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:07.732155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65125a-cbbe-47c8-9d00-72c17f4e0b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:07.754518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ec65125a-cbbe-47c8-9d00-72c17f4e0b44 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:07.757027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ec65125a-cbbe-47c8-9d00-72c17f4e0b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:07.850360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 124/241] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:07.850617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a8146-449c-40b4-8c03-7cc5a70a90e8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:07.850617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a8146-449c-40b4-8c03-7cc5a70a90e8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:07.850617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a8146-449c-40b4-8c03-7cc5a70a90e8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:07.850617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 118/242] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:07.853637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b513f3-8978-4d4b-9585-58c1645e9188 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:07.853926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b513f3-8978-4d4b-9585-58c1645e9188 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:07.854202 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b513f3-8978-4d4b-9585-58c1645e9188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:07.873499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f6b513f3-8978-4d4b-9585-58c1645e9188 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:07.874753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f6b513f3-8978-4d4b-9585-58c1645e9188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:07.928437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbe0b-17f3-4b33-a85c-52aea1a0a38a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:07.928716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbe0b-17f3-4b33-a85c-52aea1a0a38a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:07.928990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbe0b-17f3-4b33-a85c-52aea1a0a38a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:07.930372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 119/243] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:07.935764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fcb41-1334-4560-8bba-4a23e6f40a73 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:07.936114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fcb41-1334-4560-8bba-4a23e6f40a73 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:07.936381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fcb41-1334-4560-8bba-4a23e6f40a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:07.956425 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-315fcb41-1334-4560-8bba-4a23e6f40a73 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:07.956635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 125/244] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:07.958051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-315fcb41-1334-4560-8bba-4a23e6f40a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:08.009846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:08.037432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 120/245] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:08.151669 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b513f3-8978-4d4b-9585-58c1645e9188 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.171251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:08.237495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fcb41-1334-4560-8bba-4a23e6f40a73 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.259534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b513f3-8978-4d4b-9585-58c1645e9188 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.294273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5959e1-0ba3-420e-a493-e30f65ee106a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:08.294851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5959e1-0ba3-420e-a493-e30f65ee106a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:08.295129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5959e1-0ba3-420e-a493-e30f65ee106a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:08.295401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5959e1-0ba3-420e-a493-e30f65ee106a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:08.296515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5959e1-0ba3-420e-a493-e30f65ee106a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:08.297961 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5959e1-0ba3-420e-a493-e30f65ee106a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:08.299587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 126/246] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:08.305497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5959e1-0ba3-420e-a493-e30f65ee106a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.346658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fcb41-1334-4560-8bba-4a23e6f40a73 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.382421 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444e8c8-ee77-46c6-803c-f940e04714f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:08.383044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444e8c8-ee77-46c6-803c-f940e04714f8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:08.383324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444e8c8-ee77-46c6-803c-f940e04714f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:08.383593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444e8c8-ee77-46c6-803c-f940e04714f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:08.391395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:08.393463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444e8c8-ee77-46c6-803c-f940e04714f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1444e8c8-ee77-46c6-803c-f940e04714f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:08.399025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:08.400046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:08.400880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 121/247] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:08.402546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444e8c8-ee77-46c6-803c-f940e04714f8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.407626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5959e1-0ba3-420e-a493-e30f65ee106a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.436964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:08.442297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f093587-6ac8-4188-99ac-6411c0182c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:08.443108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f093587-6ac8-4188-99ac-6411c0182c8d None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:08.443418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f093587-6ac8-4188-99ac-6411c0182c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:08.443690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f093587-6ac8-4188-99ac-6411c0182c8d None admin] PATH_INFO: `/v3/groups` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:08.444408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f093587-6ac8-4188-99ac-6411c0182c8d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:08.445881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f093587-6ac8-4188-99ac-6411c0182c8d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:08.461000 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 127/248] 10.222.0.16 () {60 vars in 1134 bytes} [Mon Oct 17 10:12:08 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:12:08.462491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f093587-6ac8-4188-99ac-6411c0182c8d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.511262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444e8c8-ee77-46c6-803c-f940e04714f8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.554421 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a34de2-f898-4ac1-81ff-120c09622316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:08.554985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a34de2-f898-4ac1-81ff-120c09622316 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:08.555258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a34de2-f898-4ac1-81ff-120c09622316 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:08.555525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a34de2-f898-4ac1-81ff-120c09622316 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:08.569076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a34de2-f898-4ac1-81ff-120c09622316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e0a34de2-f898-4ac1-81ff-120c09622316 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:08.574636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:08.575569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:08.576386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:08.582506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 122/249] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:08.584814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a34de2-f898-4ac1-81ff-120c09622316 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.592052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f093587-6ac8-4188-99ac-6411c0182c8d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.635910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:08.636530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:08.636806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:08.637081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:08.637744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:08.638049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:08.639509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:08.648811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 128/250] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:08.656433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.669805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:08.710614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:08.719544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a34de2-f898-4ac1-81ff-120c09622316 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.765881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:08.767325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:08.767820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:08.768186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:08.768962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:08.769272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:08.770768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:08.780865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 123/251] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:08.787959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.788119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d792-a053-424e-b43d-cd3c5ac22caa None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.832157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160019a7-5e8b-4262-9de8-7e747648b71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:08.833635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160019a7-5e8b-4262-9de8-7e747648b71d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:08.834241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160019a7-5e8b-4262-9de8-7e747648b71d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:08.834241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160019a7-5e8b-4262-9de8-7e747648b71d None admin] PATH_INFO: `/v3/users/07abfeec43dd4275a229b0999df294d9` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:08.835349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160019a7-5e8b-4262-9de8-7e747648b71d None admin] RBAC: Authorizing `identity:get_user(user_id=07abfeec43dd4275a229b0999df294d9)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:08.836853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160019a7-5e8b-4262-9de8-7e747648b71d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:08.839259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 129/252] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:12:08 2022] GET /identity/v3/users/07abfeec43dd4275a229b0999df294d9 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:08.844747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160019a7-5e8b-4262-9de8-7e747648b71d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:08.922005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dd40d-9b03-483c-bb34-78e584c4fbb5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.941832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:08.968691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160019a7-5e8b-4262-9de8-7e747648b71d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:08.975362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc3acb-106a-4ffd-949d-b594bbe75070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:08.976056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc3acb-106a-4ffd-949d-b594bbe75070 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:08.976339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc3acb-106a-4ffd-949d-b594bbe75070 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:08.976608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc3acb-106a-4ffd-949d-b594bbe75070 None admin] PATH_INFO: `/v3/users/f217bd5b4a214ba29d98b9d4f5cd3c48` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:08.977883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc3acb-106a-4ffd-949d-b594bbe75070 None admin] RBAC: Authorizing `identity:get_user(user_id=f217bd5b4a214ba29d98b9d4f5cd3c48)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:08.979350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc3acb-106a-4ffd-949d-b594bbe75070 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:08.981493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 124/253] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:12:08 2022] GET /identity/v3/users/f217bd5b4a214ba29d98b9d4f5cd3c48 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:08.988781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc3acb-106a-4ffd-949d-b594bbe75070 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:09.013715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdf93c-60b3-48e5-a38c-5443c866e848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:09.014557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdf93c-60b3-48e5-a38c-5443c866e848 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:09.014888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdf93c-60b3-48e5-a38c-5443c866e848 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:09.015166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdf93c-60b3-48e5-a38c-5443c866e848 None admin] PATH_INFO: `/v3/system/users/07abfeec43dd4275a229b0999df294d9/roles/c1cacdb19cc543428839254e4ecf77a2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:09.016770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdf93c-60b3-48e5-a38c-5443c866e848 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=07abfeec43dd4275a229b0999df294d9, role_id=c1cacdb19cc543428839254e4ecf77a2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:09.018507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:09.018624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdf93c-60b3-48e5-a38c-5443c866e848 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:09.029999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 130/254] 10.222.0.16 () {58 vars in 1334 bytes} [Mon Oct 17 10:12:08 2022] PUT /identity/v3/system/users/07abfeec43dd4275a229b0999df294d9/roles/c1cacdb19cc543428839254e4ecf77a2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:12:09.110482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc3acb-106a-4ffd-949d-b594bbe75070 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:09.155172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2c6fc-46e3-4d19-9216-8f14ef8c3a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:09.155810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c6fc-46e3-4d19-9216-8f14ef8c3a00 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:09.156086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c6fc-46e3-4d19-9216-8f14ef8c3a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:09.156355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c6fc-46e3-4d19-9216-8f14ef8c3a00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:09.157100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c6fc-46e3-4d19-9216-8f14ef8c3a00 None admin] RBAC: Adding query filter params (role.id=cbe9824e14834e26a81ef6a408bd1bd4, user.id=f217bd5b4a214ba29d98b9d4f5cd3c48, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:09.157406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c6fc-46e3-4d19-9216-8f14ef8c3a00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:09.160459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c6fc-46e3-4d19-9216-8f14ef8c3a00 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:09.171842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 125/255] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Oct 17 10:12:08 2022] GET /identity/v3/role_assignments?user.id=f217bd5b4a214ba29d98b9d4f5cd3c48&scope.system=all&role.id=cbe9824e14834e26a81ef6a408bd1bd4 => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:10.967688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bd015-c48b-4d90-bd3d-20ea15937b1a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:10.968209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bd015-c48b-4d90-bd3d-20ea15937b1a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:10.968209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bd015-c48b-4d90-bd3d-20ea15937b1a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:10.969825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 131/256] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:10.975858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f64b27-441c-4600-8df5-cec44b408815 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:10.976312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f64b27-441c-4600-8df5-cec44b408815 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:10.976389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f64b27-441c-4600-8df5-cec44b408815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:10.998295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-43f64b27-441c-4600-8df5-cec44b408815 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:10.999652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43f64b27-441c-4600-8df5-cec44b408815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:11.077072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 126/257] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 10:12:11.379141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db665c84-3e9b-4b8a-b751-ef2502e6c6cf None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:11.379864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db665c84-3e9b-4b8a-b751-ef2502e6c6cf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:11.380483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db665c84-3e9b-4b8a-b751-ef2502e6c6cf None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:11.382370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 132/258] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:11.387645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808048a-2ffb-4777-898f-c8d95dcd03a6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:11.387929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808048a-2ffb-4777-898f-c8d95dcd03a6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:11.388202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808048a-2ffb-4777-898f-c8d95dcd03a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:11.417074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2808048a-2ffb-4777-898f-c8d95dcd03a6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:11.418261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2808048a-2ffb-4777-898f-c8d95dcd03a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:11.429932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 127/259] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 10:12:11.523968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db665c84-3e9b-4b8a-b751-ef2502e6c6cf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:11.536911 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db665c84-3e9b-4b8a-b751-ef2502e6c6cf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:11.541311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44a9bd-9645-4c97-b205-ecdcbe576ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:11.541947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44a9bd-9645-4c97-b205-ecdcbe576ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:11.542217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44a9bd-9645-4c97-b205-ecdcbe576ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:11.542515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44a9bd-9645-4c97-b205-ecdcbe576ca7 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:11.558371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44a9bd-9645-4c97-b205-ecdcbe576ca7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6d44a9bd-9645-4c97-b205-ecdcbe576ca7 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:11.563199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:11.564066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:11.564869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 133/260] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Oct 17 10:12:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:11.571038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808048a-2ffb-4777-898f-c8d95dcd03a6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:11.583030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808048a-2ffb-4777-898f-c8d95dcd03a6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:11.587638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:11.588251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:11.588524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:11.588791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:11.589469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:11.589770 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:11.591594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:12:11.591594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Oct 17 10:12:11.591594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: Oct 17 10:12:11.592506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:11.601817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 128/261] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:12:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:11.607318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44a9bd-9645-4c97-b205-ecdcbe576ca7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:11.618971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44a9bd-9645-4c97-b205-ecdcbe576ca7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:11.627541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:11.628190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:11.628463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:11.628790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] PATH_INFO: `/v3/groups/047adea22fb24ffb9c487c6f9019fa2a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:11.639166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] RBAC: Authorizing `identity:get_group(group_id=047adea22fb24ffb9c487c6f9019fa2a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:11.640891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:12:11.640891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:12:11.640891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:12:11.642108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:11.643802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 134/262] 10.222.0.16 () {56 vars in 1201 bytes} [Mon Oct 17 10:12:11 2022] GET /identity/v3/groups/047adea22fb24ffb9c487c6f9019fa2a => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:11.649759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:11.663909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e79d4a-fccd-4895-8b8e-1f1eebb80ff7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:11.672621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee747e-d7b3-4364-a5ed-d365110a1a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:11.673551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee747e-d7b3-4364-a5ed-d365110a1a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:11.674047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee747e-d7b3-4364-a5ed-d365110a1a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:11.674586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee747e-d7b3-4364-a5ed-d365110a1a40 admin admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:11.679069 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee747e-d7b3-4364-a5ed-d365110a1a40 admin admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:11.681512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee747e-d7b3-4364-a5ed-d365110a1a40 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:11.683713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 129/263] 10.222.0.16 () {56 vars in 1207 bytes} [Mon Oct 17 10:12:11 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:11.688986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:11.703957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff5579-01e6-46e0-8aaa-c9129a817b01 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:11.713671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:11.714346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:11.714640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:11.714909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/groups/047adea22fb24ffb9c487c6f9019fa2a/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:11.719624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8434c7cd8ed4e18a86583801f3af1f7, group_id=047adea22fb24ffb9c487c6f9019fa2a, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:11.720655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:12:11.720655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:12:11.720655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:12:11.722898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:11.742319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 135/264] 10.222.0.16 () {58 vars in 1463 bytes} [Mon Oct 17 10:12:11 2022] PUT /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/groups/047adea22fb24ffb9c487c6f9019fa2a/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:12.017311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee747e-d7b3-4364-a5ed-d365110a1a40 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:12.031694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee747e-d7b3-4364-a5ed-d365110a1a40 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:12.045918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:12.116529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82320091-5d45-402f-baaf-465fe1838f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:12.117176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320091-5d45-402f-baaf-465fe1838f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.117452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320091-5d45-402f-baaf-465fe1838f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.117720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320091-5d45-402f-baaf-465fe1838f54 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.133683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82320091-5d45-402f-baaf-465fe1838f54 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-82320091-5d45-402f-baaf-465fe1838f54 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:12.139045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:12.139934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:12.140715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 130/265] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Oct 17 10:12:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 124 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:12.146788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:12.156678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836bb3-6261-4ba7-8fea-25514179a9f5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.156963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836bb3-6261-4ba7-8fea-25514179a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.157233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836bb3-6261-4ba7-8fea-25514179a9f5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.158862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 131/266] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:12.164007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18f9da-f07b-4164-8335-3af283ec1f47 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.164288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18f9da-f07b-4164-8335-3af283ec1f47 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.164562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18f9da-f07b-4164-8335-3af283ec1f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.165163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd16f2e-22f5-4319-b8db-228d580d8bc3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:12.174906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ad577-e6de-41c7-9480-e61bc9318b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:12.175522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ad577-e6de-41c7-9480-e61bc9318b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.175852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ad577-e6de-41c7-9480-e61bc9318b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.176127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ad577-e6de-41c7-9480-e61bc9318b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.176795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ad577-e6de-41c7-9480-e61bc9318b18 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:12.177929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ad577-e6de-41c7-9480-e61bc9318b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:12.179103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-a48ad577-e6de-41c7-9480-e61bc9318b18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:12:12.179103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:12:12.179103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:12:12.180006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ad577-e6de-41c7-9480-e61bc9318b18 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:12.185729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1d18f9da-f07b-4164-8335-3af283ec1f47 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:12.187308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1d18f9da-f07b-4164-8335-3af283ec1f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:12.195778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 136/267] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Oct 17 10:12:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:12.197731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415c2ad-17ce-49f0-90df-6e9810b7a829 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.198192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415c2ad-17ce-49f0-90df-6e9810b7a829 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.198628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 132/268] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 10:12:12.198830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415c2ad-17ce-49f0-90df-6e9810b7a829 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.200322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 137/269] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:12.202930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18f9da-f07b-4164-8335-3af283ec1f47 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:12.212719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18f9da-f07b-4164-8335-3af283ec1f47 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:12.217208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964517e-af59-4623-b965-96423f0330ad None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.217357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:12.217907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964517e-af59-4623-b965-96423f0330ad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.218012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.218186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.218529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] PATH_INFO: `/v3/groups/047adea22fb24ffb9c487c6f9019fa2a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.218869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964517e-af59-4623-b965-96423f0330ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.219687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] RBAC: Authorizing `identity:get_group(group_id=047adea22fb24ffb9c487c6f9019fa2a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:12.220699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:12:12.220699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Oct 17 10:12:12.220699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: Oct 17 10:12:12.222494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:12.224608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 133/270] 10.222.0.16 () {56 vars in 1201 bytes} [Mon Oct 17 10:12:12 2022] GET /identity/v3/groups/047adea22fb24ffb9c487c6f9019fa2a => generated 226 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:12.230185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:12.244404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de47e52-2d16-42d0-a2eb-810266418b6f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:12.249477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582b1bc-7c3c-464b-b41d-92b5c5cf63a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:12.252659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582b1bc-7c3c-464b-b41d-92b5c5cf63a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.253098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582b1bc-7c3c-464b-b41d-92b5c5cf63a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.253573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582b1bc-7c3c-464b-b41d-92b5c5cf63a6 admin admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.254007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6964517e-af59-4623-b965-96423f0330ad None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:12.255313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6964517e-af59-4623-b965-96423f0330ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:12.255432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582b1bc-7c3c-464b-b41d-92b5c5cf63a6 admin admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:12.257837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582b1bc-7c3c-464b-b41d-92b5c5cf63a6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:12.260134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 134/271] 10.222.0.16 () {56 vars in 1207 bytes} [Mon Oct 17 10:12:12 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:12.266999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582b1bc-7c3c-464b-b41d-92b5c5cf63a6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:12.284457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582b1bc-7c3c-464b-b41d-92b5c5cf63a6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:12.294953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:12.297333 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.297783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.298196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/groups/047adea22fb24ffb9c487c6f9019fa2a/roles/eaef1c14dd23498185dccc05b242ff6a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.303009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8434c7cd8ed4e18a86583801f3af1f7, group_id=047adea22fb24ffb9c487c6f9019fa2a, role_id=eaef1c14dd23498185dccc05b242ff6a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:12.305427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:12:12.305427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Oct 17 10:12:12.305427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: Oct 17 10:12:12.310275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:12.331852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 135/272] 10.222.0.16 () {58 vars in 1463 bytes} [Mon Oct 17 10:12:12 2022] PUT /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7/groups/047adea22fb24ffb9c487c6f9019fa2a/roles/eaef1c14dd23498185dccc05b242ff6a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:12.373479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 138/273] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:12.747117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:12.811287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964517e-af59-4623-b965-96423f0330ad None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:12.837335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5a08b-7b36-4603-bd2f-5def4c242716 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:12.842416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a956f-9cec-40f2-bf29-ebcffb438f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:12.843291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a956f-9cec-40f2-bf29-ebcffb438f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.843767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a956f-9cec-40f2-bf29-ebcffb438f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.844568 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a956f-9cec-40f2-bf29-ebcffb438f1a admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.857657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:12.874100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a956f-9cec-40f2-bf29-ebcffb438f1a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-487a956f-9cec-40f2-bf29-ebcffb438f1a admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:12.884823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:12.885778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:12.886625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:12.892698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 136/274] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Oct 17 10:12:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 140 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:12.894633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94325-553b-466c-8281-b39030c2f588 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.894915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94325-553b-466c-8281-b39030c2f588 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.895191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94325-553b-466c-8281-b39030c2f588 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.897439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 137/275] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:12.917739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab94325-553b-466c-8281-b39030c2f588 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:12.932088 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab94325-553b-466c-8281-b39030c2f588 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:12.937121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11be368-69c4-4705-995b-0fd9fccc7a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:12.937719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11be368-69c4-4705-995b-0fd9fccc7a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.937993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11be368-69c4-4705-995b-0fd9fccc7a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.938262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11be368-69c4-4705-995b-0fd9fccc7a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.938965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11be368-69c4-4705-995b-0fd9fccc7a29 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:12.939262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11be368-69c4-4705-995b-0fd9fccc7a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:12.940858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11be368-69c4-4705-995b-0fd9fccc7a29 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:12.954503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 138/276] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:12:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:12.955846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79dec7-b08c-4818-99b2-3143c3ad7db7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:12.956127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79dec7-b08c-4818-99b2-3143c3ad7db7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:12.956398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79dec7-b08c-4818-99b2-3143c3ad7db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:12.980577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4a79dec7-b08c-4818-99b2-3143c3ad7db7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:12.981756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4a79dec7-b08c-4818-99b2-3143c3ad7db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:12.990203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 139/277] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 10:12:12.991195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79dec7-b08c-4818-99b2-3143c3ad7db7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.005628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79dec7-b08c-4818-99b2-3143c3ad7db7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.015159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da258ef-323c-4196-97ac-0eaa746da06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.015969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da258ef-323c-4196-97ac-0eaa746da06c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.016244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da258ef-323c-4196-97ac-0eaa746da06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.017422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da258ef-323c-4196-97ac-0eaa746da06c admin admin] PATH_INFO: `/v3/groups/047adea22fb24ffb9c487c6f9019fa2a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.019388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da258ef-323c-4196-97ac-0eaa746da06c admin admin] RBAC: Authorizing `identity:get_group(group_id=047adea22fb24ffb9c487c6f9019fa2a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.023083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da258ef-323c-4196-97ac-0eaa746da06c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.025550 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 140/278] 10.222.0.16 () {56 vars in 1201 bytes} [Mon Oct 17 10:12:12 2022] GET /identity/v3/groups/047adea22fb24ffb9c487c6f9019fa2a => generated 226 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:13.028487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964517e-af59-4623-b965-96423f0330ad None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.031514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da258ef-323c-4196-97ac-0eaa746da06c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.050946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da258ef-323c-4196-97ac-0eaa746da06c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.061606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e57cce-b403-4782-8bf3-5fb65e89daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.063877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e57cce-b403-4782-8bf3-5fb65e89daa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.064603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e57cce-b403-4782-8bf3-5fb65e89daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.064878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e57cce-b403-4782-8bf3-5fb65e89daa1 admin admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.067060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e57cce-b403-4782-8bf3-5fb65e89daa1 admin admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.082689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e57cce-b403-4782-8bf3-5fb65e89daa1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.087363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 141/279] 10.222.0.16 () {56 vars in 1207 bytes} [Mon Oct 17 10:12:13 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:13.092619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e57cce-b403-4782-8bf3-5fb65e89daa1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.106366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965760b3-d506-40f0-b9b7-992f186b9bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.108265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965760b3-d506-40f0-b9b7-992f186b9bda None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.108542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965760b3-d506-40f0-b9b7-992f186b9bda None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.108813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965760b3-d506-40f0-b9b7-992f186b9bda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.109760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e57cce-b403-4782-8bf3-5fb65e89daa1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.119593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578325f-0f95-44b2-8d30-e93693523312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.120245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578325f-0f95-44b2-8d30-e93693523312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.120515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578325f-0f95-44b2-8d30-e93693523312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.121760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578325f-0f95-44b2-8d30-e93693523312 admin admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185/groups/047adea22fb24ffb9c487c6f9019fa2a/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.124904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578325f-0f95-44b2-8d30-e93693523312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102d784de27145ad9641bd35a5ea7185, group_id=047adea22fb24ffb9c487c6f9019fa2a, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.130200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578325f-0f95-44b2-8d30-e93693523312 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.133446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965760b3-d506-40f0-b9b7-992f186b9bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-965760b3-d506-40f0-b9b7-992f186b9bda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:13.140162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:13.141066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 10:12:13.141880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:13.142507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 139/280] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:13.148146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965760b3-d506-40f0-b9b7-992f186b9bda None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.162124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 142/281] 10.222.0.16 () {58 vars in 1463 bytes} [Mon Oct 17 10:12:13 2022] PUT /identity/v3/projects/102d784de27145ad9641bd35a5ea7185/groups/047adea22fb24ffb9c487c6f9019fa2a/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:13.263456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:13.279268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965760b3-d506-40f0-b9b7-992f186b9bda None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.325384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.326005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.326281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.326590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.327268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:13.327568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.329007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.339224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 140/282] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:13.345061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578325f-0f95-44b2-8d30-e93693523312 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.385043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:13.478146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578325f-0f95-44b2-8d30-e93693523312 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.515708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd925d-344b-45bc-a0a7-3ba4e6bff6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.516263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd925d-344b-45bc-a0a7-3ba4e6bff6af None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.516541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd925d-344b-45bc-a0a7-3ba4e6bff6af None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.516820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd925d-344b-45bc-a0a7-3ba4e6bff6af None admin] PATH_INFO: `/v3/users/07abfeec43dd4275a229b0999df294d9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.517949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd925d-344b-45bc-a0a7-3ba4e6bff6af None admin] RBAC: Authorizing `identity:get_user(user_id=07abfeec43dd4275a229b0999df294d9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.519421 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd925d-344b-45bc-a0a7-3ba4e6bff6af None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.521871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 143/283] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:12:13 2022] GET /identity/v3/users/07abfeec43dd4275a229b0999df294d9 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:13.527931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.642521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd925d-344b-45bc-a0a7-3ba4e6bff6af None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.647616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0bd6-6e0b-441a-99d5-d90128652f37 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.691033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.691649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.691940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.692213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.693378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] RBAC: Adding query filter params (role.id=c1cacdb19cc543428839254e4ecf77a2, user.id=07abfeec43dd4275a229b0999df294d9, scope.system=all) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:13.693489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.696511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.718220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 141/284] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Oct 17 10:12:13 2022] GET /identity/v3/role_assignments?user.id=07abfeec43dd4275a229b0999df294d9&scope.system=all&role.id=c1cacdb19cc543428839254e4ecf77a2 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:13.734673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd925d-344b-45bc-a0a7-3ba4e6bff6af None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.749185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0eb59e-181b-4aa6-9165-3217062216de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.750185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb59e-181b-4aa6-9165-3217062216de admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.750855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb59e-181b-4aa6-9165-3217062216de admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.751428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb59e-181b-4aa6-9165-3217062216de admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.777192 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0eb59e-181b-4aa6-9165-3217062216de admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7b0eb59e-181b-4aa6-9165-3217062216de admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:13.782395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:13.783273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:13.784097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:13.786852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 144/285] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Oct 17 10:12:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:13.792697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.811532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb299a47-bb2f-40f6-8c52-7d4ce6f059bb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.821505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.822130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.822431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.822703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.823392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:13.823692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.825328 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.828175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:13.840311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 142/286] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Oct 17 10:12:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:13.847487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0eb59e-181b-4aa6-9165-3217062216de admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.861726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0eb59e-181b-4aa6-9165-3217062216de admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.867767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5d5c7-da5c-4729-8913-e0447e2051c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.868763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5d5c7-da5c-4729-8913-e0447e2051c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.869268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5d5c7-da5c-4729-8913-e0447e2051c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.869751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5d5c7-da5c-4729-8913-e0447e2051c0 admin admin] PATH_INFO: `/v3/groups/047adea22fb24ffb9c487c6f9019fa2a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.872591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5d5c7-da5c-4729-8913-e0447e2051c0 admin admin] RBAC: Authorizing `identity:get_group(group_id=047adea22fb24ffb9c487c6f9019fa2a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.874967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5d5c7-da5c-4729-8913-e0447e2051c0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.876862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 145/287] 10.222.0.16 () {56 vars in 1201 bytes} [Mon Oct 17 10:12:13 2022] GET /identity/v3/groups/047adea22fb24ffb9c487c6f9019fa2a => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:13.882883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.900208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e733-d8de-4f1b-82bd-de696bd3d6b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.908026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1851b-e6c1-4172-a283-09bd3a275c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.909028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1851b-e6c1-4172-a283-09bd3a275c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.909509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1851b-e6c1-4172-a283-09bd3a275c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.910055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1851b-e6c1-4172-a283-09bd3a275c80 admin admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.912119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1851b-e6c1-4172-a283-09bd3a275c80 admin admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.914468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1851b-e6c1-4172-a283-09bd3a275c80 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.916449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 143/288] 10.222.0.16 () {56 vars in 1207 bytes} [Mon Oct 17 10:12:13 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:13.922319 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5d5c7-da5c-4729-8913-e0447e2051c0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:13.932898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5d5c7-da5c-4729-8913-e0447e2051c0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:13.941620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233fa5-2f7c-45eb-b814-a9c75c091dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:13.943279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233fa5-2f7c-45eb-b814-a9c75c091dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:13.943766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233fa5-2f7c-45eb-b814-a9c75c091dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:13.944318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233fa5-2f7c-45eb-b814-a9c75c091dd9 admin admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185/groups/047adea22fb24ffb9c487c6f9019fa2a/roles/eaef1c14dd23498185dccc05b242ff6a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:13.951370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233fa5-2f7c-45eb-b814-a9c75c091dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102d784de27145ad9641bd35a5ea7185, group_id=047adea22fb24ffb9c487c6f9019fa2a, role_id=eaef1c14dd23498185dccc05b242ff6a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:13.954534 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233fa5-2f7c-45eb-b814-a9c75c091dd9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:13.980450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 146/289] 10.222.0.16 () {58 vars in 1463 bytes} [Mon Oct 17 10:12:13 2022] PUT /identity/v3/projects/102d784de27145ad9641bd35a5ea7185/groups/047adea22fb24ffb9c487c6f9019fa2a/roles/eaef1c14dd23498185dccc05b242ff6a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:14.087231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a33ef-f1fa-4bc5-bf44-1bf1d3b01bbd None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:14.087820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a33ef-f1fa-4bc5-bf44-1bf1d3b01bbd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:14.088310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a33ef-f1fa-4bc5-bf44-1bf1d3b01bbd None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:14.089956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 144/290] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:14.095910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4afd-fecf-408e-b071-d38752187611 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:14.096366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4afd-fecf-408e-b071-d38752187611 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:14.096466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4afd-fecf-408e-b071-d38752187611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:14.124228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-619f4afd-fecf-408e-b071-d38752187611 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:14.125425 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-619f4afd-fecf-408e-b071-d38752187611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:14.147774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:14.213064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 147/291] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 10:12:14.540840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a33ef-f1fa-4bc5-bf44-1bf1d3b01bbd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:14.556577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a33ef-f1fa-4bc5-bf44-1bf1d3b01bbd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:14.561847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0e429-17d3-4aa2-9fb5-a324331e2b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:14.562523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0e429-17d3-4aa2-9fb5-a324331e2b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:14.562805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0e429-17d3-4aa2-9fb5-a324331e2b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:14.563079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0e429-17d3-4aa2-9fb5-a324331e2b03 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:14.580362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0e429-17d3-4aa2-9fb5-a324331e2b03 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-48d0e429-17d3-4aa2-9fb5-a324331e2b03 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:14.585786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:14.586744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:14.587579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 145/292] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Oct 17 10:12:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:14.592882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f4afd-fecf-408e-b071-d38752187611 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:14.605606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f4afd-fecf-408e-b071-d38752187611 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:14.612006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:14.612617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:14.612896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:14.613166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:14.613847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:14.614145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:14.616304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:14.628231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 148/293] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Oct 17 10:12:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:14.633954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0e429-17d3-4aa2-9fb5-a324331e2b03 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:14.644469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0e429-17d3-4aa2-9fb5-a324331e2b03 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:14.653262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5fd06-0b56-4686-bc47-8cfe7eb43a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:14.653895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5fd06-0b56-4686-bc47-8cfe7eb43a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:14.654173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5fd06-0b56-4686-bc47-8cfe7eb43a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:14.654472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5fd06-0b56-4686-bc47-8cfe7eb43a5c admin admin] PATH_INFO: `/v3/groups/0a4082c25deb494ebc383d3827e23b48` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:14.662107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5fd06-0b56-4686-bc47-8cfe7eb43a5c admin admin] RBAC: Authorizing `identity:get_group(group_id=0a4082c25deb494ebc383d3827e23b48)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:14.664086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5fd06-0b56-4686-bc47-8cfe7eb43a5c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:14.665924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 146/294] 10.222.0.16 () {56 vars in 1201 bytes} [Mon Oct 17 10:12:14 2022] GET /identity/v3/groups/0a4082c25deb494ebc383d3827e23b48 => generated 229 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:14.671576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:14.684605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3420f8-f39b-43d6-a2e9-f91f9827f42c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:14.689160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64940e33-0327-4a6e-a3e8-fb6ca0d75a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:14.689853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64940e33-0327-4a6e-a3e8-fb6ca0d75a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:14.690134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64940e33-0327-4a6e-a3e8-fb6ca0d75a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:14.690431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64940e33-0327-4a6e-a3e8-fb6ca0d75a24 admin admin] PATH_INFO: `/v3/projects/e6150f4298a94f959838ad2351eaefb9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:14.691634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64940e33-0327-4a6e-a3e8-fb6ca0d75a24 admin admin] RBAC: Authorizing `identity:get_project(project_id=e6150f4298a94f959838ad2351eaefb9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:14.694292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64940e33-0327-4a6e-a3e8-fb6ca0d75a24 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:14.695957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 149/295] 10.222.0.16 () {56 vars in 1207 bytes} [Mon Oct 17 10:12:14 2022] GET /identity/v3/projects/e6150f4298a94f959838ad2351eaefb9 => generated 344 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:14.701765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5fd06-0b56-4686-bc47-8cfe7eb43a5c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:14.712932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5fd06-0b56-4686-bc47-8cfe7eb43a5c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:14.721688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8566-055f-490b-a2b5-111271173255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:14.722646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8566-055f-490b-a2b5-111271173255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:14.722925 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8566-055f-490b-a2b5-111271173255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:14.723200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8566-055f-490b-a2b5-111271173255 admin admin] PATH_INFO: `/v3/projects/e6150f4298a94f959838ad2351eaefb9/groups/0a4082c25deb494ebc383d3827e23b48/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:14.731083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa8566-055f-490b-a2b5-111271173255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6150f4298a94f959838ad2351eaefb9, group_id=0a4082c25deb494ebc383d3827e23b48, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:14.733827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa8566-055f-490b-a2b5-111271173255 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:14.753385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 147/296] 10.222.0.16 () {58 vars in 1463 bytes} [Mon Oct 17 10:12:14 2022] PUT /identity/v3/projects/e6150f4298a94f959838ad2351eaefb9/groups/0a4082c25deb494ebc383d3827e23b48/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:15.109383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c9f8-b89e-435a-ae83-189410977fde None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:15.110028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c9f8-b89e-435a-ae83-189410977fde None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:15.110028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c9f8-b89e-435a-ae83-189410977fde None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:15.111450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 150/297] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:15.117672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10de2-e16e-4144-be7c-3e372f38d89a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:15.117960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10de2-e16e-4144-be7c-3e372f38d89a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:15.118239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10de2-e16e-4144-be7c-3e372f38d89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:15.139926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-54b10de2-e16e-4144-be7c-3e372f38d89a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:15.141119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-54b10de2-e16e-4144-be7c-3e372f38d89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:15.219300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 148/298] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:15.505706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3c9f8-b89e-435a-ae83-189410977fde None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:15.621246 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3c9f8-b89e-435a-ae83-189410977fde None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:15.655494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c2b3-63f6-4d52-9828-28639dc5dfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:15.656123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c2b3-63f6-4d52-9828-28639dc5dfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:15.656398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c2b3-63f6-4d52-9828-28639dc5dfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:15.656663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c2b3-63f6-4d52-9828-28639dc5dfc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:15.670017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742c2b3-63f6-4d52-9828-28639dc5dfc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c742c2b3-63f6-4d52-9828-28639dc5dfc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:15.675124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:15.676104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:15.677165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 151/299] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:15.682966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b10de2-e16e-4144-be7c-3e372f38d89a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:15.782816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:15.794080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b10de2-e16e-4144-be7c-3e372f38d89a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:15.830591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae9e0-0ae2-4c45-9143-7c495ad58bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:15.831246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae9e0-0ae2-4c45-9143-7c495ad58bea None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:15.831527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae9e0-0ae2-4c45-9143-7c495ad58bea None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:15.831794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae9e0-0ae2-4c45-9143-7c495ad58bea None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:15.832466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae9e0-0ae2-4c45-9143-7c495ad58bea None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:15.832763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae9e0-0ae2-4c45-9143-7c495ad58bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:15.834840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae9e0-0ae2-4c45-9143-7c495ad58bea None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:15.843836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 149/300] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:15.849999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c2b3-63f6-4d52-9828-28639dc5dfc8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:15.851305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6fff2-590e-454d-8cde-0b51b7388e23 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:15.851585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6fff2-590e-454d-8cde-0b51b7388e23 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:15.851859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6fff2-590e-454d-8cde-0b51b7388e23 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:15.853252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 150/301] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:15.864831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92144d-bf2c-4208-abd8-4509e2a59374 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:15.864831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92144d-bf2c-4208-abd8-4509e2a59374 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:15.864831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92144d-bf2c-4208-abd8-4509e2a59374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:15.877735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:15.882174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ee92144d-bf2c-4208-abd8-4509e2a59374 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:15.884720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ee92144d-bf2c-4208-abd8-4509e2a59374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:16.002376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 151/302] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:16.034027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c2b3-63f6-4d52-9828-28639dc5dfc8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:16.084524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b346f-d9f3-412d-8511-4b44b00f2337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:16.085157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b346f-d9f3-412d-8511-4b44b00f2337 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:16.085441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b346f-d9f3-412d-8511-4b44b00f2337 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:16.085712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b346f-d9f3-412d-8511-4b44b00f2337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:16.165121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b346f-d9f3-412d-8511-4b44b00f2337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:16.166811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b346f-d9f3-412d-8511-4b44b00f2337 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:16.180987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:16.280194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 152/303] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 10:12:16.285802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92144d-bf2c-4208-abd8-4509e2a59374 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:16.338404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb8713-2c57-4d89-ad58-ada6613c898f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:16.338953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb8713-2c57-4d89-ad58-ada6613c898f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:16.339233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb8713-2c57-4d89-ad58-ada6613c898f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:16.340831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 153/304] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:16.345574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370071-547f-48da-ab84-46cbf67b9b14 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:16.346013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370071-547f-48da-ab84-46cbf67b9b14 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:16.346495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370071-547f-48da-ab84-46cbf67b9b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:16.374168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-97370071-547f-48da-ab84-46cbf67b9b14 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:16.375628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97370071-547f-48da-ab84-46cbf67b9b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:16.414501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:16.435281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92144d-bf2c-4208-abd8-4509e2a59374 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:16.471961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 154/305] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:16.473397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97370071-547f-48da-ab84-46cbf67b9b14 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:16.486383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f7b2-2d49-4999-b5f2-a701129184d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:16.487026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f7b2-2d49-4999-b5f2-a701129184d2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:16.487302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f7b2-2d49-4999-b5f2-a701129184d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:16.487570 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f7b2-2d49-4999-b5f2-a701129184d2 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:16.488692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f7b2-2d49-4999-b5f2-a701129184d2 None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:16.490760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f7b2-2d49-4999-b5f2-a701129184d2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:16.492991 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 152/306] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:12:16 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:16.498270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f7b2-2d49-4999-b5f2-a701129184d2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:16.549765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:16.620890 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f7b2-2d49-4999-b5f2-a701129184d2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:16.665025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e789b2a-971e-4f18-9eef-c5c1200e1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:16.665692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e789b2a-971e-4f18-9eef-c5c1200e1d2c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:16.666082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e789b2a-971e-4f18-9eef-c5c1200e1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:16.666492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e789b2a-971e-4f18-9eef-c5c1200e1d2c None admin] PATH_INFO: `/v3/groups/047adea22fb24ffb9c487c6f9019fa2a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:16.669333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e789b2a-971e-4f18-9eef-c5c1200e1d2c None admin] RBAC: Authorizing `identity:get_group(group_id=047adea22fb24ffb9c487c6f9019fa2a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:16.669333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e789b2a-971e-4f18-9eef-c5c1200e1d2c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:16.671884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 153/307] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Oct 17 10:12:16 2022] GET /identity/v3/groups/047adea22fb24ffb9c487c6f9019fa2a => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:16.679061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e789b2a-971e-4f18-9eef-c5c1200e1d2c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:16.680354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97370071-547f-48da-ab84-46cbf67b9b14 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:16.704614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:16.744828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af3a33-cdde-4ab3-8dcb-1d29826e38ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:16.745471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af3a33-cdde-4ab3-8dcb-1d29826e38ff None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:16.745744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af3a33-cdde-4ab3-8dcb-1d29826e38ff None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:16.746011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af3a33-cdde-4ab3-8dcb-1d29826e38ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:16.769110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af3a33-cdde-4ab3-8dcb-1d29826e38ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b2af3a33-cdde-4ab3-8dcb-1d29826e38ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:16.775844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:16.776777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:16.777607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:16.778466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 155/308] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:12:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:16.783101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af3a33-cdde-4ab3-8dcb-1d29826e38ff None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:16.865660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e789b2a-971e-4f18-9eef-c5c1200e1d2c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:16.921779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:16.922744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:16.923168 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:16.923605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:16.924479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] RBAC: Adding query filter params (group.id=047adea22fb24ffb9c487c6f9019fa2a, role.id=cbe9824e14834e26a81ef6a408bd1bd4, scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:16.925085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:16.927918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:16.941885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 154/309] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Oct 17 10:12:16 2022] GET /identity/v3/role_assignments?group.id=047adea22fb24ffb9c487c6f9019fa2a&scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7&role.id=cbe9824e14834e26a81ef6a408bd1bd4 => generated 611 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:16.960871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:16.975824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af3a33-cdde-4ab3-8dcb-1d29826e38ff None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:16.979605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:17.040180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:17.040922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.041160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.041342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.044429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:17.044429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:17.044429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:17.064064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 156/310] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:17.076913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:17.137866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:17.167709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19cf50-a145-4f68-b086-de924fbf3860 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:17.213366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413c73-7033-4596-9063-ab866164961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:17.214503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413c73-7033-4596-9063-ab866164961c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.214974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413c73-7033-4596-9063-ab866164961c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.215261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413c73-7033-4596-9063-ab866164961c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.235648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75413c73-7033-4596-9063-ab866164961c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-75413c73-7033-4596-9063-ab866164961c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:17.241789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:17.242777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:17.245148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 155/311] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:17.249111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d11af-a69a-4bf2-9eaf-9f420195a65d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:17.250286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413c73-7033-4596-9063-ab866164961c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:17.294444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeff4f5-f9a5-42cf-9867-cca5e050b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:17.295064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeff4f5-f9a5-42cf-9867-cca5e050b832 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.295340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeff4f5-f9a5-42cf-9867-cca5e050b832 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.295612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeff4f5-f9a5-42cf-9867-cca5e050b832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.334833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:17.374923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeff4f5-f9a5-42cf-9867-cca5e050b832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:17.376502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeff4f5-f9a5-42cf-9867-cca5e050b832 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:17.376740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413c73-7033-4596-9063-ab866164961c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:17.417028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f5f65-35b6-4b9e-ae28-c7f3b4c28e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:17.417555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5f65-35b6-4b9e-ae28-c7f3b4c28e7c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.417829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5f65-35b6-4b9e-ae28-c7f3b4c28e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.418099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5f65-35b6-4b9e-ae28-c7f3b4c28e7c None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.418795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f5f65-35b6-4b9e-ae28-c7f3b4c28e7c None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:17.419097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f5f65-35b6-4b9e-ae28-c7f3b4c28e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:17.420473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f5f65-35b6-4b9e-ae28-c7f3b4c28e7c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:17.433890 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 156/312] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:12:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:17.440160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9baa-019e-4016-bb9b-0ae8db7e6d4f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.440449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9baa-019e-4016-bb9b-0ae8db7e6d4f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.440726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9baa-019e-4016-bb9b-0ae8db7e6d4f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.442212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 157/313] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:17.443501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a9baa-019e-4016-bb9b-0ae8db7e6d4f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:17.456822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:17.508244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 157/314] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 10:12:17.509500 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941eb65-2397-4f48-94a3-7b358ecd810e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.509783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941eb65-2397-4f48-94a3-7b358ecd810e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.510055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941eb65-2397-4f48-94a3-7b358ecd810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.534787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6941eb65-2397-4f48-94a3-7b358ecd810e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:17.536609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6941eb65-2397-4f48-94a3-7b358ecd810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:17.573736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a9baa-019e-4016-bb9b-0ae8db7e6d4f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:17.614990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:17.622352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df69eeb-a1b5-457d-a638-f2c1343cc62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:17.622979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df69eeb-a1b5-457d-a638-f2c1343cc62e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.623258 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df69eeb-a1b5-457d-a638-f2c1343cc62e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.623532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df69eeb-a1b5-457d-a638-f2c1343cc62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.630761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 158/315] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:17.631969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941eb65-2397-4f48-94a3-7b358ecd810e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:17.714460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df69eeb-a1b5-457d-a638-f2c1343cc62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:17.716732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df69eeb-a1b5-457d-a638-f2c1343cc62e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:17.728920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:17.765137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941eb65-2397-4f48-94a3-7b358ecd810e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:17.812425 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44058299-67ad-457d-a370-5dbe155a5286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:17.813399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44058299-67ad-457d-a370-5dbe155a5286 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.813922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44058299-67ad-457d-a370-5dbe155a5286 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.814413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44058299-67ad-457d-a370-5dbe155a5286 None admin] PATH_INFO: `/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.815755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44058299-67ad-457d-a370-5dbe155a5286 None admin] RBAC: Authorizing `identity:get_project(project_id=d8434c7cd8ed4e18a86583801f3af1f7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:17.817891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44058299-67ad-457d-a370-5dbe155a5286 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:17.820309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 159/316] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:12:17 2022] GET /identity/v3/projects/d8434c7cd8ed4e18a86583801f3af1f7 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:17.825498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44058299-67ad-457d-a370-5dbe155a5286 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:17.851039 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 158/317] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:12:17.859922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b54d0-26d8-42fa-b75e-2e1aa1b24370 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:17.860198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b54d0-26d8-42fa-b75e-2e1aa1b24370 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:17.860464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b54d0-26d8-42fa-b75e-2e1aa1b24370 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:17.861776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 159/318] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:17.862495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b54d0-26d8-42fa-b75e-2e1aa1b24370 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:17.914380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:17.961804 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44058299-67ad-457d-a370-5dbe155a5286 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:17.991261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b54d0-26d8-42fa-b75e-2e1aa1b24370 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:17.999021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:18.001684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a299360-4eff-4f9e-a6fc-3f7183c4791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.002646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a299360-4eff-4f9e-a6fc-3f7183c4791b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.002756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a299360-4eff-4f9e-a6fc-3f7183c4791b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.002986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a299360-4eff-4f9e-a6fc-3f7183c4791b None admin] PATH_INFO: `/v3/groups/047adea22fb24ffb9c487c6f9019fa2a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.005752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a299360-4eff-4f9e-a6fc-3f7183c4791b None admin] RBAC: Authorizing `identity:get_group(group_id=047adea22fb24ffb9c487c6f9019fa2a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:18.006953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a299360-4eff-4f9e-a6fc-3f7183c4791b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:18.008879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 160/319] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Oct 17 10:12:17 2022] GET /identity/v3/groups/047adea22fb24ffb9c487c6f9019fa2a => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:18.010043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688eb70-f43a-4188-a19e-3f6470f39d08 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.010388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688eb70-f43a-4188-a19e-3f6470f39d08 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.010623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688eb70-f43a-4188-a19e-3f6470f39d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.030746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1688eb70-f43a-4188-a19e-3f6470f39d08 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:18.031931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1688eb70-f43a-4188-a19e-3f6470f39d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:18.036544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ec827-b74d-474b-bb8c-70ddf7812bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.037195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec827-b74d-474b-bb8c-70ddf7812bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.037453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec827-b74d-474b-bb8c-70ddf7812bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.037725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec827-b74d-474b-bb8c-70ddf7812bd6 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.040361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ec827-b74d-474b-bb8c-70ddf7812bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:18.040472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ec827-b74d-474b-bb8c-70ddf7812bd6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:18.042243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 160/320] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:12:17 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:18.042875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ec827-b74d-474b-bb8c-70ddf7812bd6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:18.126864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 161/321] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:18.128005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688eb70-f43a-4188-a19e-3f6470f39d08 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:18.159635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ec827-b74d-474b-bb8c-70ddf7812bd6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:18.202935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e46103-9a5a-463f-b406-e65370140f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.203736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46103-9a5a-463f-b406-e65370140f33 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.204021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46103-9a5a-463f-b406-e65370140f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.204291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46103-9a5a-463f-b406-e65370140f33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.213668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:18.220690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e46103-9a5a-463f-b406-e65370140f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f8e46103-9a5a-463f-b406-e65370140f33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:18.229334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:18.230276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:18.231128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 161/322] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:18.231790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e46103-9a5a-463f-b406-e65370140f33 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:18.268960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688eb70-f43a-4188-a19e-3f6470f39d08 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:18.277740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:18.313028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.313628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.313898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.314160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.314908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] RBAC: Adding query filter params (group.id=047adea22fb24ffb9c487c6f9019fa2a, role.id=eaef1c14dd23498185dccc05b242ff6a, scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:18.315210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:18.316621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:18.327219 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 162/323] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/role_assignments?group.id=047adea22fb24ffb9c487c6f9019fa2a&scope.project.id=d8434c7cd8ed4e18a86583801f3af1f7&role.id=eaef1c14dd23498185dccc05b242ff6a => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:18.328317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:18.375068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e46103-9a5a-463f-b406-e65370140f33 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:18.424086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af92ed-5a40-4df6-a371-a7279dd498d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.425216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af92ed-5a40-4df6-a371-a7279dd498d1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.425493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af92ed-5a40-4df6-a371-a7279dd498d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.425813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af92ed-5a40-4df6-a371-a7279dd498d1 None admin] PATH_INFO: `/v3/groups/047adea22fb24ffb9c487c6f9019fa2a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.427739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af92ed-5a40-4df6-a371-a7279dd498d1 None admin] RBAC: Authorizing `identity:get_group(group_id=047adea22fb24ffb9c487c6f9019fa2a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:18.429353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af92ed-5a40-4df6-a371-a7279dd498d1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:18.431892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 162/324] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/groups/047adea22fb24ffb9c487c6f9019fa2a => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:18.432862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af92ed-5a40-4df6-a371-a7279dd498d1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:18.457632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccf3f9-b3f8-41ef-9730-1d62c71913fc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:18.495713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:18.510449 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.511093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.511355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.511626 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.512459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:18.512644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:18.514007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:18.525869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 163/325] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:18.527062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:18.569869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af92ed-5a40-4df6-a371-a7279dd498d1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:18.571607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:18.613635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d4149-75d4-4885-8890-1d071637e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.614384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d4149-75d4-4885-8890-1d071637e26d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.614708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d4149-75d4-4885-8890-1d071637e26d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.615006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d4149-75d4-4885-8890-1d071637e26d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.625791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d4149-75d4-4885-8890-1d071637e26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-461d4149-75d4-4885-8890-1d071637e26d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:18.631882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:18.632802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:18.633616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 163/326] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:18.634311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d4149-75d4-4885-8890-1d071637e26d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:18.663281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043bf99-ed40-4a2b-8218-cf3eae2d0825 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:18.708663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.709267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.709550 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.709822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.710657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] RBAC: Adding query filter params (group.id=047adea22fb24ffb9c487c6f9019fa2a, role.id=cbe9824e14834e26a81ef6a408bd1bd4, scope.project.id=102d784de27145ad9641bd35a5ea7185) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:18.710966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:18.712401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:18.720004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 164/327] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/role_assignments?group.id=047adea22fb24ffb9c487c6f9019fa2a&scope.project.id=102d784de27145ad9641bd35a5ea7185&role.id=cbe9824e14834e26a81ef6a408bd1bd4 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:18.721264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:18.759309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d4149-75d4-4885-8890-1d071637e26d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:18.794120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:18.809166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e955b6-a467-43f0-a06f-062f59873458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.809794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e955b6-a467-43f0-a06f-062f59873458 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.810070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e955b6-a467-43f0-a06f-062f59873458 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.810364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e955b6-a467-43f0-a06f-062f59873458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.850311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:18.873925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ef443-49e9-4b60-8bb2-451f126f880a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:18.900765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e955b6-a467-43f0-a06f-062f59873458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:18.902874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e955b6-a467-43f0-a06f-062f59873458 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:18.926661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:18.927324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:18.927987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:18.928118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:18.928725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:18.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:18.931377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:18.941733 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 165/328] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:18.948171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:19.033331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 164/329] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:12:19.039393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e955b6-a467-43f0-a06f-062f59873458 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:19.073703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d3b90-06da-415a-a64d-d042d51b3e86 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:19.090550 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:19.113340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:19.114134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add22b8d-933c-4ca7-b014-585c79742d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:19.115710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add22b8d-933c-4ca7-b014-585c79742d11 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:19.115931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add22b8d-933c-4ca7-b014-585c79742d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:19.116412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add22b8d-933c-4ca7-b014-585c79742d11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:19.163728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e955b6-a467-43f0-a06f-062f59873458 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:19.197696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add22b8d-933c-4ca7-b014-585c79742d11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:19.199167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add22b8d-933c-4ca7-b014-585c79742d11 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:19.199534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9857f49-f604-4e9a-97e4-a28eb55df8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:19.200161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9857f49-f604-4e9a-97e4-a28eb55df8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:19.200440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9857f49-f604-4e9a-97e4-a28eb55df8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:19.200706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9857f49-f604-4e9a-97e4-a28eb55df8e3 None admin] PATH_INFO: `/v3/projects/102d784de27145ad9641bd35a5ea7185` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:19.201820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9857f49-f604-4e9a-97e4-a28eb55df8e3 None admin] RBAC: Authorizing `identity:get_project(project_id=102d784de27145ad9641bd35a5ea7185)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:19.203284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9857f49-f604-4e9a-97e4-a28eb55df8e3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:19.204970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 165/330] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:12:19 2022] GET /identity/v3/projects/102d784de27145ad9641bd35a5ea7185 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:19.211006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9857f49-f604-4e9a-97e4-a28eb55df8e3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:19.296724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 166/331] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 10:12:19.302222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add22b8d-933c-4ca7-b014-585c79742d11 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:19.308958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9857f49-f604-4e9a-97e4-a28eb55df8e3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:19.315011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:19.337898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:19.343991 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f54a1-909b-4e70-abbd-1fd673be8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:19.344605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f54a1-909b-4e70-abbd-1fd673be8cef None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:19.344878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f54a1-909b-4e70-abbd-1fd673be8cef None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:19.345148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f54a1-909b-4e70-abbd-1fd673be8cef None admin] PATH_INFO: `/v3/groups/047adea22fb24ffb9c487c6f9019fa2a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:19.346221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f54a1-909b-4e70-abbd-1fd673be8cef None admin] RBAC: Authorizing `identity:get_group(group_id=047adea22fb24ffb9c487c6f9019fa2a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:19.347933 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f54a1-909b-4e70-abbd-1fd673be8cef None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:19.349499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 166/332] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Oct 17 10:12:19 2022] GET /identity/v3/groups/047adea22fb24ffb9c487c6f9019fa2a => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:19.355211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f54a1-909b-4e70-abbd-1fd673be8cef None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:19.400375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add22b8d-933c-4ca7-b014-585c79742d11 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:19.434271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9621a4-b8f3-477d-9de6-7543ae0c9ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:19.434915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9621a4-b8f3-477d-9de6-7543ae0c9ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:19.435186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9621a4-b8f3-477d-9de6-7543ae0c9ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:19.435455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9621a4-b8f3-477d-9de6-7543ae0c9ea9 None admin] PATH_INFO: `/v3/projects/e6150f4298a94f959838ad2351eaefb9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:19.436556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9621a4-b8f3-477d-9de6-7543ae0c9ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=e6150f4298a94f959838ad2351eaefb9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:19.437963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9621a4-b8f3-477d-9de6-7543ae0c9ea9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:19.439626 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 167/333] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:12:19 2022] GET /identity/v3/projects/e6150f4298a94f959838ad2351eaefb9 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:19.445510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9621a4-b8f3-477d-9de6-7543ae0c9ea9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:19.452866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f54a1-909b-4e70-abbd-1fd673be8cef None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:19.485861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:19.486594 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:19.486871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:19.487139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:19.487851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] RBAC: Adding query filter params (group.id=047adea22fb24ffb9c487c6f9019fa2a, role.id=eaef1c14dd23498185dccc05b242ff6a, scope.project.id=102d784de27145ad9641bd35a5ea7185) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:19.488148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:19.489642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:19.495268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 167/334] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Oct 17 10:12:19 2022] GET /identity/v3/role_assignments?group.id=047adea22fb24ffb9c487c6f9019fa2a&scope.project.id=102d784de27145ad9641bd35a5ea7185&role.id=eaef1c14dd23498185dccc05b242ff6a => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:19.534648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:19.555814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9621a4-b8f3-477d-9de6-7543ae0c9ea9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:19.591350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9ef78-833d-4b39-9b61-9f83755d85ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:19.591886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9ef78-833d-4b39-9b61-9f83755d85ef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:19.592155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9ef78-833d-4b39-9b61-9f83755d85ef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:19.592417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9ef78-833d-4b39-9b61-9f83755d85ef None admin] PATH_INFO: `/v3/groups/0a4082c25deb494ebc383d3827e23b48` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:19.593528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9ef78-833d-4b39-9b61-9f83755d85ef None admin] RBAC: Authorizing `identity:get_group(group_id=0a4082c25deb494ebc383d3827e23b48)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:19.594989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9ef78-833d-4b39-9b61-9f83755d85ef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:19.596583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 168/335] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Oct 17 10:12:19 2022] GET /identity/v3/groups/0a4082c25deb494ebc383d3827e23b48 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:19.603160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:19.669513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:19.710040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a81875-fa88-4299-aa9c-f07ed36b255f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:19.744501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13f339-b828-4fd5-849c-76b70ee50962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:19.745105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f339-b828-4fd5-849c-76b70ee50962 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:19.745383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f339-b828-4fd5-849c-76b70ee50962 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:19.745654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f339-b828-4fd5-849c-76b70ee50962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:19.746416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13f339-b828-4fd5-849c-76b70ee50962 None admin] RBAC: Adding query filter params (group.id=0a4082c25deb494ebc383d3827e23b48, role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=e6150f4298a94f959838ad2351eaefb9) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:19.746724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13f339-b828-4fd5-849c-76b70ee50962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:19.748159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13f339-b828-4fd5-849c-76b70ee50962 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:19.753970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 168/336] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Oct 17 10:12:19 2022] GET /identity/v3/role_assignments?group.id=0a4082c25deb494ebc383d3827e23b48&scope.project.id=e6150f4298a94f959838ad2351eaefb9&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:22.351914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8228-3843-4138-8ed1-b1588076052c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:22.352422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8228-3843-4138-8ed1-b1588076052c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:22.352422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8228-3843-4138-8ed1-b1588076052c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:22.353934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 169/337] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:22.359437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2fb3d-4776-4505-b61e-644257df1585 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:22.359867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2fb3d-4776-4505-b61e-644257df1585 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:22.359962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2fb3d-4776-4505-b61e-644257df1585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:22.378613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2fe2fb3d-4776-4505-b61e-644257df1585 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:22.380346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2fe2fb3d-4776-4505-b61e-644257df1585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:22.468844 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 169/338] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:22.775879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57357151-e43c-496f-995d-b8363a9220e5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:22.776551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57357151-e43c-496f-995d-b8363a9220e5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:22.777020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57357151-e43c-496f-995d-b8363a9220e5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:22.778902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 170/339] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:22.784401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4776c2-6f55-4625-a656-beaf79e50829 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:22.784916 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4776c2-6f55-4625-a656-beaf79e50829 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:22.785377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4776c2-6f55-4625-a656-beaf79e50829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:22.815582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7a4776c2-6f55-4625-a656-beaf79e50829 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:22.817155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7a4776c2-6f55-4625-a656-beaf79e50829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:22.884891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:22.904040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57357151-e43c-496f-995d-b8363a9220e5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:22.904406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 170/340] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:23.001432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993616d3-d3fb-4910-9ec0-8d8a7cdf6754 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.001968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993616d3-d3fb-4910-9ec0-8d8a7cdf6754 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.002423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993616d3-d3fb-4910-9ec0-8d8a7cdf6754 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.004177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 171/341] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:23.012348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cdf70-ffed-40fa-b1c6-11bdd605974c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.012836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cdf70-ffed-40fa-b1c6-11bdd605974c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.013303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cdf70-ffed-40fa-b1c6-11bdd605974c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.015040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 172/342] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:23.016276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9f1ee-cdd2-49ab-b0e7-10b21a378fad None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.016750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9f1ee-cdd2-49ab-b0e7-10b21a378fad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.017215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9f1ee-cdd2-49ab-b0e7-10b21a378fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.037181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-63a9f1ee-cdd2-49ab-b0e7-10b21a378fad None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:23.039486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-63a9f1ee-cdd2-49ab-b0e7-10b21a378fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:23.050912 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57357151-e43c-496f-995d-b8363a9220e5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:23.098259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:23.100008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.100676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.100676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.111364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:23.113036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:23.123493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:23.132389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:23.133567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:23.134701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:23.134701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:23.134701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:23.136682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 173/343] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:23.138142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3e581-3c2d-4528-94a8-e7681c2de578 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.138589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3e581-3c2d-4528-94a8-e7681c2de578 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.138998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3e581-3c2d-4528-94a8-e7681c2de578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.141376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 171/344] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:23.142236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:23.161768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ddb3e581-3c2d-4528-94a8-e7681c2de578 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:23.163755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ddb3e581-3c2d-4528-94a8-e7681c2de578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:23.279707 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 174/345] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:23.281044 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3e581-3c2d-4528-94a8-e7681c2de578 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:23.294416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e975a-62e7-4a51-8785-32f57cd7571d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:23.346284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:23.346905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.347177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.347706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.348340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:23.349257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:23.350658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:23.375466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:23.383909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 172/346] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:23.390820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:23.484249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3e581-3c2d-4528-94a8-e7681c2de578 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:23.498498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:23.524376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:23.525504 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.525995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.526520 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.537984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:23.539834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:23.544356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e172ce3-7e37-4d88-86e9-0b95d3e69aaf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:23.556705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:23.557670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:23.558590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:23.558590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:23.558590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:23.560450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 175/347] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:23.561426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:23.588614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefaf36-c61e-43c3-b9b4-9a5efd2778b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:23.589225 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefaf36-c61e-43c3-b9b4-9a5efd2778b0 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.589502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefaf36-c61e-43c3-b9b4-9a5efd2778b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.589769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefaf36-c61e-43c3-b9b4-9a5efd2778b0 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.590454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefaf36-c61e-43c3-b9b4-9a5efd2778b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:23.591897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefaf36-c61e-43c3-b9b4-9a5efd2778b0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:23.619120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 173/348] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:12:23 2022] POST /identity/v3/users => generated 246 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:12:23.620257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefaf36-c61e-43c3-b9b4-9a5efd2778b0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:23.683395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:23.690111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c1c4e-0257-4435-b6cb-8ee62bfdac5c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:23.731660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:23.732301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.732577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.732844 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.745997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:23.747575 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:23.764872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:23.765876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:23.772774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:23.772774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:23.772774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:23.772774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 176/349] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:23.772774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:23.794720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefaf36-c61e-43c3-b9b4-9a5efd2778b0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:23.816435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:23.845565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:23.846470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.846758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.847011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.847918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:23.848217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:23.851837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:23.863687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 174/350] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:23.864740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:23.904329 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fe5c6-f45f-454c-a4aa-6a028cec1668 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:23.942843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:23.943908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:23.944564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:23.945038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:23.959706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:23.962261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:23.979797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:23.980746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:23.981592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:23.981592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:23.981592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:23.981757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382559d8-2101-4883-8964-a1857ce7e47b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:23.981835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 177/351] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:23.982487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:23.984660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:24.024289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ea506-51cf-4998-acb1-81be26bc8dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:24.024901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea506-51cf-4998-acb1-81be26bc8dbb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:24.025199 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea506-51cf-4998-acb1-81be26bc8dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:24.025443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea506-51cf-4998-acb1-81be26bc8dbb None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:24.026140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ea506-51cf-4998-acb1-81be26bc8dbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:24.026477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ea506-51cf-4998-acb1-81be26bc8dbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:24.027844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ea506-51cf-4998-acb1-81be26bc8dbb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:24.043789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 175/352] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:24.043789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087179f7-020a-4c48-b9d3-acf5d6bd5527 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:24.043789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087179f7-020a-4c48-b9d3-acf5d6bd5527 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:24.043789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087179f7-020a-4c48-b9d3-acf5d6bd5527 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:24.043789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 176/353] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:24.043789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087179f7-020a-4c48-b9d3-acf5d6bd5527 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:24.091328 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:24.099123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568ca26-c8f1-4268-a67b-41f0113a1e15 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:24.134708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c412343-b6f6-4725-b01d-1a296f9adca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:24.135323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c412343-b6f6-4725-b01d-1a296f9adca6 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:24.135603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c412343-b6f6-4725-b01d-1a296f9adca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:24.135880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c412343-b6f6-4725-b01d-1a296f9adca6 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:24.136525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c412343-b6f6-4725-b01d-1a296f9adca6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:24.138043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c412343-b6f6-4725-b01d-1a296f9adca6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:24.160235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 178/354] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:12:23 2022] POST /identity/v3/users => generated 248 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:24.167480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087179f7-020a-4c48-b9d3-acf5d6bd5527 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:24.171370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c412343-b6f6-4725-b01d-1a296f9adca6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:24.216728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724e5e-58d2-4182-9a98-f2b8ecfdc74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:24.217651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724e5e-58d2-4182-9a98-f2b8ecfdc74e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:24.217938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724e5e-58d2-4182-9a98-f2b8ecfdc74e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:24.218202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724e5e-58d2-4182-9a98-f2b8ecfdc74e None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:24.218903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724e5e-58d2-4182-9a98-f2b8ecfdc74e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:24.219201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724e5e-58d2-4182-9a98-f2b8ecfdc74e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:24.220604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724e5e-58d2-4182-9a98-f2b8ecfdc74e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:24.242365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 177/355] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:24.242365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656284-0659-448c-8c29-381c122cc46c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:24.242365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656284-0659-448c-8c29-381c122cc46c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:24.242365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656284-0659-448c-8c29-381c122cc46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:24.258676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:24.265981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-71656284-0659-448c-8c29-381c122cc46c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:24.269660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-71656284-0659-448c-8c29-381c122cc46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:24.318233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c412343-b6f6-4725-b01d-1a296f9adca6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:24.363564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac1260-4144-4df5-944e-df88d5013eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:24.364106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac1260-4144-4df5-944e-df88d5013eb4 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:24.364904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac1260-4144-4df5-944e-df88d5013eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:24.365145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 178/356] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:24.365351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac1260-4144-4df5-944e-df88d5013eb4 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:24.365874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71656284-0659-448c-8c29-381c122cc46c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:24.366037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac1260-4144-4df5-944e-df88d5013eb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:24.367529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac1260-4144-4df5-944e-df88d5013eb4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:24.399307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 179/357] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:12:24 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:24.441745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:24.508429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71656284-0659-448c-8c29-381c122cc46c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:24.564309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c718b-342a-4710-a02e-908d2cb2c028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:24.564932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c718b-342a-4710-a02e-908d2cb2c028 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:24.565206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c718b-342a-4710-a02e-908d2cb2c028 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:24.565472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c718b-342a-4710-a02e-908d2cb2c028 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:24.566129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c718b-342a-4710-a02e-908d2cb2c028 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:24.568684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c718b-342a-4710-a02e-908d2cb2c028 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:24.610212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 179/358] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:12:24 2022] POST /identity/v3/users => generated 249 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:24.688462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac1260-4144-4df5-944e-df88d5013eb4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:24.810249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac1260-4144-4df5-944e-df88d5013eb4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:24.826815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:24.852410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:24.853022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:24.853301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:24.853572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:24.854256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:24.855679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:24.866222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.sql.core [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 10:12:24.866222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 10:12:24.866222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [parameters: {'id': 'd98af297b3b54eaab7856822a56cf243', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 10:12:24.866222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81407) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application result.read() Oct 17 10:12:24.882794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return ref.to_dict() Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:12:24.883799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application yield resource Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise self.value Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application session.commit() Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 10:12:24.885207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.session.flush() Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application compat.raise_( Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 10:12:24.886304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:12:24.887229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application result.read() Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:12:24.889673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [parameters: {'id': 'd98af297b3b54eaab7856822a56cf243', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:24.895013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 10:12:24.896586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:24.897598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 180/359] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Oct 17 10:12:24 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:24.897687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c718b-342a-4710-a02e-908d2cb2c028 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:24.984914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:25.005285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c718b-342a-4710-a02e-908d2cb2c028 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:25.040583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84cde1-1dbb-4509-af7a-be7cc4430431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:25.041316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84cde1-1dbb-4509-af7a-be7cc4430431 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:25.041661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84cde1-1dbb-4509-af7a-be7cc4430431 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:25.041996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84cde1-1dbb-4509-af7a-be7cc4430431 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:25.053581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84cde1-1dbb-4509-af7a-be7cc4430431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4c84cde1-1dbb-4509-af7a-be7cc4430431 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:25.059029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:25.060138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 10:12:25.061162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:25.063194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 180/360] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:12:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:25.068281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:25.209305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474442d9-4524-41b8-98b8-22916882e69a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:25.255618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fceeb-3792-41a1-9d67-c4a9720472e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:25.256243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fceeb-3792-41a1-9d67-c4a9720472e5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:25.256515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fceeb-3792-41a1-9d67-c4a9720472e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:25.256780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fceeb-3792-41a1-9d67-c4a9720472e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:25.257447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fceeb-3792-41a1-9d67-c4a9720472e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:25.257743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fceeb-3792-41a1-9d67-c4a9720472e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:25.259138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fceeb-3792-41a1-9d67-c4a9720472e5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:25.267973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 181/361] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:26.170724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0719-28b2-48e7-9dd7-248b5a75e5a3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:26.171241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0719-28b2-48e7-9dd7-248b5a75e5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:26.171241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0719-28b2-48e7-9dd7-248b5a75e5a3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:26.172872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 181/362] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:26.177951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0d95-707b-473f-864b-5f6d892d28c2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:26.178394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0d95-707b-473f-864b-5f6d892d28c2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:26.178567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0d95-707b-473f-864b-5f6d892d28c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:26.203891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-557a0d95-707b-473f-864b-5f6d892d28c2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:26.205082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-557a0d95-707b-473f-864b-5f6d892d28c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:26.250306 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:26.300904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 182/363] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:26.682461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a0719-28b2-48e7-9dd7-248b5a75e5a3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:26.705378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a743e4c-659e-44a6-99c3-dfddde8ac328 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:26.705710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a743e4c-659e-44a6-99c3-dfddde8ac328 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:26.705910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a743e4c-659e-44a6-99c3-dfddde8ac328 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:26.707512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 183/364] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:26.716880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6fd41-3b2c-4507-882c-2f8213314174 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:26.717149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6fd41-3b2c-4507-882c-2f8213314174 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:26.717416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6fd41-3b2c-4507-882c-2f8213314174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:26.750490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3af6fd41-3b2c-4507-882c-2f8213314174 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:26.751748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3af6fd41-3b2c-4507-882c-2f8213314174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:26.854209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a0719-28b2-48e7-9dd7-248b5a75e5a3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:26.888917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 184/365] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:26.902569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf598309-b239-4fc8-87c9-afbad5275595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:26.903261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf598309-b239-4fc8-87c9-afbad5275595 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:26.903540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf598309-b239-4fc8-87c9-afbad5275595 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:26.903806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf598309-b239-4fc8-87c9-afbad5275595 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:26.928153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf598309-b239-4fc8-87c9-afbad5275595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bf598309-b239-4fc8-87c9-afbad5275595 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:26.933071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:26.933953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:26.935403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 182/366] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:26.940014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6fd41-3b2c-4507-882c-2f8213314174 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:27.091852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:27.094275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6fd41-3b2c-4507-882c-2f8213314174 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:27.147915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:27.148525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.148795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.149060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.149719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:27.150012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:27.151410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:27.160088 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 185/367] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:27.165404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf598309-b239-4fc8-87c9-afbad5275595 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:27.168222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:27.221964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:27.338952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf598309-b239-4fc8-87c9-afbad5275595 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:27.375999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488becb7-7a7e-4b01-8d5c-64024c4e8336 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:27.389879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c47564-1b55-4b90-9e96-7eb42872be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:27.390576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c47564-1b55-4b90-9e96-7eb42872be9a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.390850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c47564-1b55-4b90-9e96-7eb42872be9a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.391114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c47564-1b55-4b90-9e96-7eb42872be9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.402502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c47564-1b55-4b90-9e96-7eb42872be9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:27.403946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c47564-1b55-4b90-9e96-7eb42872be9a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:27.417139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef78f9a-134b-4e78-8231-7c8f29a65da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:27.417765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef78f9a-134b-4e78-8231-7c8f29a65da4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.418038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef78f9a-134b-4e78-8231-7c8f29a65da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.418309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef78f9a-134b-4e78-8231-7c8f29a65da4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d1c47564-1b55-4b90-9e96-7eb42872be9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:27.419062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:27.419964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:27.420828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:27.420828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:27.420828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:27.420828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 183/368] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:27.421744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b8198-6e5a-45a5-950c-469bdcdd8918 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.421744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b8198-6e5a-45a5-950c-469bdcdd8918 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.421915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b8198-6e5a-45a5-950c-469bdcdd8918 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.423404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 184/369] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:27.428899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b8198-6e5a-45a5-950c-469bdcdd8918 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:27.434109 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef78f9a-134b-4e78-8231-7c8f29a65da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6ef78f9a-134b-4e78-8231-7c8f29a65da4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:27.439684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:27.440606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:27.441418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 186/370] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:27.443252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c1e8-84c6-4ae6-9eea-65e0a0f9f7b9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.447795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c1e8-84c6-4ae6-9eea-65e0a0f9f7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.448072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c1e8-84c6-4ae6-9eea-65e0a0f9f7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.466620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5389c1e8-84c6-4ae6-9eea-65e0a0f9f7b9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:27.467796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5389c1e8-84c6-4ae6-9eea-65e0a0f9f7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:27.486074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:27.500858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:27.538173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 187/371] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:27.539046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389c1e8-84c6-4ae6-9eea-65e0a0f9f7b9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:27.558141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b8198-6e5a-45a5-950c-469bdcdd8918 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:27.600086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42649b-4dbd-4b7a-a4c0-f63c5c23abd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:27.601159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42649b-4dbd-4b7a-a4c0-f63c5c23abd2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.601646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42649b-4dbd-4b7a-a4c0-f63c5c23abd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.602163 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42649b-4dbd-4b7a-a4c0-f63c5c23abd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.603039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42649b-4dbd-4b7a-a4c0-f63c5c23abd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:27.603553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42649b-4dbd-4b7a-a4c0-f63c5c23abd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:27.606175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42649b-4dbd-4b7a-a4c0-f63c5c23abd2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:27.621592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 185/372] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:27.623974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1bb39-aba9-47e9-ace9-1587b8a496a2 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.624457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1bb39-aba9-47e9-ace9-1587b8a496a2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.624957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1bb39-aba9-47e9-ace9-1587b8a496a2 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.626655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 186/373] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:27.631359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1bb39-aba9-47e9-ace9-1587b8a496a2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:27.648210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389c1e8-84c6-4ae6-9eea-65e0a0f9f7b9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:27.685014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710d246-f247-42dc-8d8c-14f335839625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:27.685625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710d246-f247-42dc-8d8c-14f335839625 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.685965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710d246-f247-42dc-8d8c-14f335839625 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.686289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710d246-f247-42dc-8d8c-14f335839625 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.687061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710d246-f247-42dc-8d8c-14f335839625 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:27.687365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710d246-f247-42dc-8d8c-14f335839625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:27.688870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710d246-f247-42dc-8d8c-14f335839625 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:27.702357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 188/374] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:27.703641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1a68-5b4a-4c9a-95ee-f40c2db9f537 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.703921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1a68-5b4a-4c9a-95ee-f40c2db9f537 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.704189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1a68-5b4a-4c9a-95ee-f40c2db9f537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.724356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ed2d1a68-5b4a-4c9a-95ee-f40c2db9f537 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:27.725534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ed2d1a68-5b4a-4c9a-95ee-f40c2db9f537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:27.735679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:27.758443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1bb39-aba9-47e9-ace9-1587b8a496a2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:27.776480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:27.799449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 189/375] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:27.800593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1a68-5b4a-4c9a-95ee-f40c2db9f537 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:27.809701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:27.810350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.810661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.810923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:27.819241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:27.820791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:27.836916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:27.837958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:12:27.838929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:27.842919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 187/376] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Oct 17 10:12:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:27.843779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:27.948518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1a68-5b4a-4c9a-95ee-f40c2db9f537 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:27.969670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ad0ff-9043-4b4b-b6bd-55ccf912675f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:27.997260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:27.997883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:27.998521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:27.998521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.009940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.011355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.014046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.014793 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9515d06-644d-4e07-837d-e8c946032c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.015681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9515d06-644d-4e07-837d-e8c946032c4d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.016047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9515d06-644d-4e07-837d-e8c946032c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.016490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9515d06-644d-4e07-837d-e8c946032c4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.029035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9515d06-644d-4e07-837d-e8c946032c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:28.029726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:28.030845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:28.031688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:28.031688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:28.031688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d9515d06-644d-4e07-837d-e8c946032c4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:28.038228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:28.039280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:28.040117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 188/378] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:28.041311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 190/378] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:28.042540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.043280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9515d06-644d-4e07-837d-e8c946032c4d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.087813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.158069 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9515d06-644d-4e07-837d-e8c946032c4d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.158519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb4aff-e251-4294-964c-fe4bc2a09c97 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.194441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.194809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.195038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.195295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.195391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.195554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.195657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.195822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.196267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:28.196442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:28.196590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.196743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.197970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.198062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.209753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 191/379] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:28.210568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.235459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 189/380] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:28.235459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.272186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.315333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab515e-96ce-4da9-9f9b-49f405933aad None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.327118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.336113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab736357-4089-481c-b42c-43f48e4c1a9c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.349289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c71c64-7809-4292-a0b2-5764e0dd1a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.350100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c71c64-7809-4292-a0b2-5764e0dd1a1d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.350394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c71c64-7809-4292-a0b2-5764e0dd1a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.350662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c71c64-7809-4292-a0b2-5764e0dd1a1d None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.351313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c71c64-7809-4292-a0b2-5764e0dd1a1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:28.351611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c71c64-7809-4292-a0b2-5764e0dd1a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.352962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c71c64-7809-4292-a0b2-5764e0dd1a1d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.360482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 192/381] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:28.361599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45199445-9755-449d-9869-0203370e86ea None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.361874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45199445-9755-449d-9869-0203370e86ea None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.362143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45199445-9755-449d-9869-0203370e86ea None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.363346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 193/382] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:28.364022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45199445-9755-449d-9869-0203370e86ea None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.375483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76fa0d-f5f6-469b-bf87-1e9164c59b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.376645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76fa0d-f5f6-469b-bf87-1e9164c59b14 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.376923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76fa0d-f5f6-469b-bf87-1e9164c59b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.377186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76fa0d-f5f6-469b-bf87-1e9164c59b14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.387991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76fa0d-f5f6-469b-bf87-1e9164c59b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ce76fa0d-f5f6-469b-bf87-1e9164c59b14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:28.392888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:28.393784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:28.394658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:28.399112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 190/383] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:28.399880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76fa0d-f5f6-469b-bf87-1e9164c59b14 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.464569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45199445-9755-449d-9869-0203370e86ea None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.481983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.497694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76fa0d-f5f6-469b-bf87-1e9164c59b14 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.499355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.499976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.500260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.500533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.507531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.508979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:28.522490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:28.523438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 10:12:28.524352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:28.528276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 194/384] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:28.529103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.532273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35370edf-197f-4f4e-a8d1-9919ef26d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.532896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370edf-197f-4f4e-a8d1-9919ef26d768 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.533165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370edf-197f-4f4e-a8d1-9919ef26d768 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.533428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370edf-197f-4f4e-a8d1-9919ef26d768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.543930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35370edf-197f-4f4e-a8d1-9919ef26d768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.545372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35370edf-197f-4f4e-a8d1-9919ef26d768 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-35370edf-197f-4f4e-a8d1-9919ef26d768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:28.559963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:28.565478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:28.566755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:28.566755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:28.566755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:28.566755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 191/385] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:28.566755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395082-e383-40b1-a8aa-8ff572d456e1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.566755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395082-e383-40b1-a8aa-8ff572d456e1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.566755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395082-e383-40b1-a8aa-8ff572d456e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.570919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.581437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e5395082-e383-40b1-a8aa-8ff572d456e1 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:28.582644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e5395082-e383-40b1-a8aa-8ff572d456e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:28.640795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a522a-c2b9-4c11-b6b3-f93b5aa4e196 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.648541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 192/386] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:28.649371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395082-e383-40b1-a8aa-8ff572d456e1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.674843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.675465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.675740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.676009 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.687391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.688845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:28.704217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:28.705104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:28.706051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:28.706051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:28.706051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:28.707517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 195/387] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:28.708623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.741143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.762190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395082-e383-40b1-a8aa-8ff572d456e1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.798547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.799197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.799454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.799720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.800478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:28.800782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.802884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.804607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.809184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e5128-f1ea-444a-bddf-9545429cdb77 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.817009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 193/388] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:28.817009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.845519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.846120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.846421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.846694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.847675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:28.847979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.849750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.891217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 196/389] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:28.891217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.934203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b466e-c53f-4943-bb22-1e83976de8c9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:28.968032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:28.968629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:28.968905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:28.969171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:28.969805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:28.970100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:28.971491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:28.981618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:28.984434 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 194/390] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:28.986041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:28.994595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98041b53-109a-44a0-a525-17a1dac8a26b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.027580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.028177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.028452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.028720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.029358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:29.029681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.031070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.040577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.042766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 197/391] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:29.043546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.090008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80c681-9d62-4a24-9ce4-5ac17c0aec32 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.125061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.125689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.125965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.126239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.136515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.138098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.144770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a599-66b1-4642-97a4-fde80b4886b7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:29.152980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:29.153948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:29.155064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:29.155064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:29.155064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:29.155064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 195/392] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:29.155064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.183253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.183862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.184140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.184408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.194805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.196260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.205029 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:29.212481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:29.213568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:29.214592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:29.214592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:29.214592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:29.218602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 198/393] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:29.219380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.255418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9ae1c-4fc9-4596-a359-98dc1611e8e3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.264187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.289898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.290555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.290827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.291091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.301280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.302772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:29.316383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:29.317412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:29.318485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:29.318485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:29.318485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:29.318485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 196/394] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:29.318485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.321431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f418a-2192-401f-ab65-98fb5dfb2c70 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.356113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.356659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.356932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.357195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.363630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.365610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:29.377005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:29.378037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 199/395] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:29.379348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.422477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403e585-16d8-424f-9d48-052d69d9721f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.437241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.457430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.458595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.458869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.459139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] PATH_INFO: `/v3/users/glance` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.466515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.467948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.478014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac7cf4-0d73-4a5f-ba5b-18a013b692ba None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:29.479491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:29.480511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 197/396] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:29.481419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.503976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.517011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.517618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.517896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.518164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.518830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:29.519133 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.521091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.532776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 200/397] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:29.533560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.582341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bee2d-4395-45be-b390-3e7913ef0ccd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.616309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be268b80-ac13-4117-8410-735b01820665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.616897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268b80-ac13-4117-8410-735b01820665 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.617183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268b80-ac13-4117-8410-735b01820665 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.617473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268b80-ac13-4117-8410-735b01820665 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.618108 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be268b80-ac13-4117-8410-735b01820665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:29.618431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be268b80-ac13-4117-8410-735b01820665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.619794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be268b80-ac13-4117-8410-735b01820665 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.631460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 198/398] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:29.632240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be268b80-ac13-4117-8410-735b01820665 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.632457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e63f3-2a53-4b57-9428-36d6bdec2887 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.654317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.668862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.669460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.669739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.670016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.670691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:29.671000 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.672373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.684283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 201/399] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:29.686076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.710231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.731222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be268b80-ac13-4117-8410-735b01820665 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.764209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.764783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.765055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.765320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.766001 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:29.766295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.767732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.783001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dae106-0e8d-44a1-89c4-44aaccc781af None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.784600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 199/400] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:29.785386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.823071 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.823665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.823937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.824320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.825043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:29.825336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.826802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.843904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 202/401] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:29.844689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.884511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.886653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949d5d8-0d47-417b-a1a5-5d38bcb92697 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.920432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.921045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.921317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.921585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.928567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.929983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:29.932544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:29.942097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:29.943738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 10:12:29.944658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:29.945320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d965d84-523a-4b2e-a321-9d8740225b36 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:29.945393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 200/402] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:29.945393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:29.981559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:29.982174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:29.982481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:29.982750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:29.989984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:29.991458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:30.002218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:30.003149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 203/403] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:30.009222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.044725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e7155-15bf-42a4-8287-8a899f3209cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.081826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab675d-e878-459d-a7d8-b87cdc328459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.082467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab675d-e878-459d-a7d8-b87cdc328459 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.082746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab675d-e878-459d-a7d8-b87cdc328459 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.083686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab675d-e878-459d-a7d8-b87cdc328459 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.084431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab675d-e878-459d-a7d8-b87cdc328459 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.085863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab675d-e878-459d-a7d8-b87cdc328459 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.107012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 201/404] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:12:29 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:30.107619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab675d-e878-459d-a7d8-b87cdc328459 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.109351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:30.130068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade9d63-7321-47e1-b895-3f1425071760 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.169597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:30.171831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b75be-0452-4b2b-8d5a-381f4ec3a337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.173329 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b75be-0452-4b2b-8d5a-381f4ec3a337 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.174071 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b75be-0452-4b2b-8d5a-381f4ec3a337 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.174525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b75be-0452-4b2b-8d5a-381f4ec3a337 None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/9d52624718de4457adc44e9965cd7a09/roles/4045fa40bdd54e9f889b47e1fb2704f5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.191931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b75be-0452-4b2b-8d5a-381f4ec3a337 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=9d52624718de4457adc44e9965cd7a09, role_id=4045fa40bdd54e9f889b47e1fb2704f5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.193969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b75be-0452-4b2b-8d5a-381f4ec3a337 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.214380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 204/405] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:12:30 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/9d52624718de4457adc44e9965cd7a09/roles/4045fa40bdd54e9f889b47e1fb2704f5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:30.214380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b75be-0452-4b2b-8d5a-381f4ec3a337 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.234474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab675d-e878-459d-a7d8-b87cdc328459 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.283892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.284641 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.285156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.285429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.300920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.303355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:30.320030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:30.321195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:30.322072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:30.322072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:30.322072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:30.322072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 202/406] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:30.322310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.352167 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b75be-0452-4b2b-8d5a-381f4ec3a337 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.401783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.402460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.402736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.403004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.403737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:30.404046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.405528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.430683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 205/407] 10.222.0.16 () {56 vars in 1139 bytes} [Mon Oct 17 10:12:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:30.431749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.441737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:30.458360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feea3cd-c609-494a-8b32-ad3909e0fb2b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.465549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:30.498776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.499873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.500142 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.500405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.501359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:30.501656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.503786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.521619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 203/408] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:30.522503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.548690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4897a-2a42-4432-ac33-62e68620094b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.583697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.584494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.584874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.585246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.585988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:30.586413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.587950 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.604793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 206/409] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:30.605798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.648854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ec944-6f36-4954-a20a-81a679dcd372 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.687079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:30.690986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.691613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.691883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.692687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.704713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.705855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:30.722939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:30.724383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:30.725253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:30.725253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:30.725253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:30.729896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 204/410] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:30.730848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.738510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e60c9-b7d6-405d-a0b6-5d3893753e44 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.755389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:30.790200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59809f6-e946-4867-848e-6c86264f3742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.791118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59809f6-e946-4867-848e-6c86264f3742 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.791606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59809f6-e946-4867-848e-6c86264f3742 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.792016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59809f6-e946-4867-848e-6c86264f3742 None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/24f1a6c05ed94f5fa81dd355497b6754/roles/4045fa40bdd54e9f889b47e1fb2704f5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.817135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59809f6-e946-4867-848e-6c86264f3742 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=24f1a6c05ed94f5fa81dd355497b6754, role_id=4045fa40bdd54e9f889b47e1fb2704f5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.819633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59809f6-e946-4867-848e-6c86264f3742 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.839292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 207/411] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:12:30 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/24f1a6c05ed94f5fa81dd355497b6754/roles/4045fa40bdd54e9f889b47e1fb2704f5 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:30.841106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59809f6-e946-4867-848e-6c86264f3742 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.842756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c800e-7afe-4f66-bb67-12550912c2eb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:30.896780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:30.897811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:30.898612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:30.899057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:30.908878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:30.911226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:30.928366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:30.929468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:30.930544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:30.930544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:30.930544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:30.930544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:30.930544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:30.930544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:30.930544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:30.930544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:30.932695 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 205/412] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:30.939268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:30.979804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59809f6-e946-4867-848e-6c86264f3742 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:31.014596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:31.055221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:31.055994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:31.056363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:31.056728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:31.057491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:31.058731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:31.060510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:31.092716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 208/413] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:31.093859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:31.116813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:31.141798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd76f3a-d00b-474c-ac9d-efea2c8b40ef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:31.190471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:31.191349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:31.191887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:31.192980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:31.193983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:31.194506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:31.196838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:31.217058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 206/414] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:31.224239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:31.235602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb342ac4-499c-4468-89c8-fe5e9b9dfb1f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:31.299354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:31.299983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:31.300257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:31.300522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:31.314253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:31.315740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:31.336482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:31.337363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:31.338213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:31.338213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:31.338213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:31.338213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:31.338213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:31.338213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:31.338213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:31.338213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:31.346507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 209/415] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:31.351282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:31.388373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089bfee-00ba-479c-a540-a3dfde972dc6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:31.424530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c58cb5-dbbf-426b-8dcd-743095f77a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:31.425624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58cb5-dbbf-426b-8dcd-743095f77a0d None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:31.425911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58cb5-dbbf-426b-8dcd-743095f77a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:31.426179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58cb5-dbbf-426b-8dcd-743095f77a0d None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/9ac934ed0122470cb489d6b324c664f6/roles/4045fa40bdd54e9f889b47e1fb2704f5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:31.440239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:31.446947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c58cb5-dbbf-426b-8dcd-743095f77a0d None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=9ac934ed0122470cb489d6b324c664f6, role_id=4045fa40bdd54e9f889b47e1fb2704f5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:31.448792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c58cb5-dbbf-426b-8dcd-743095f77a0d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:31.476037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:31.481603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 207/416] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:12:31 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/9ac934ed0122470cb489d6b324c664f6/roles/4045fa40bdd54e9f889b47e1fb2704f5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:31.600271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273837-b4e5-4b80-b6c1-23a3af25537a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:31.648658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446fd1e-db81-4cf3-a526-89fc4b732e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:31.649293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446fd1e-db81-4cf3-a526-89fc4b732e84 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:31.649574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446fd1e-db81-4cf3-a526-89fc4b732e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:31.649844 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446fd1e-db81-4cf3-a526-89fc4b732e84 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:31.650590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446fd1e-db81-4cf3-a526-89fc4b732e84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:31.650900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446fd1e-db81-4cf3-a526-89fc4b732e84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:31.652390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446fd1e-db81-4cf3-a526-89fc4b732e84 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:31.680775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 210/417] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:31.686922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c58cb5-dbbf-426b-8dcd-743095f77a0d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:31.839050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c58cb5-dbbf-426b-8dcd-743095f77a0d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:31.896056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccc07f-befa-4b59-88c7-1ae2081dd1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:31.897105 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc07f-befa-4b59-88c7-1ae2081dd1ba None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:31.897582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc07f-befa-4b59-88c7-1ae2081dd1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:31.898068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc07f-befa-4b59-88c7-1ae2081dd1ba None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/bd7883e3f6ab4f58969f51ffd439b1c0/roles/4045fa40bdd54e9f889b47e1fb2704f5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:31.922141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccc07f-befa-4b59-88c7-1ae2081dd1ba None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=bd7883e3f6ab4f58969f51ffd439b1c0, role_id=4045fa40bdd54e9f889b47e1fb2704f5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:31.923902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccc07f-befa-4b59-88c7-1ae2081dd1ba None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:31.949554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 208/418] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:12:31 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/bd7883e3f6ab4f58969f51ffd439b1c0/roles/4045fa40bdd54e9f889b47e1fb2704f5 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:32.628053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b480b95-0f48-467e-8c14-33bfced7e925 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:32.628582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b480b95-0f48-467e-8c14-33bfced7e925 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:32.628582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b480b95-0f48-467e-8c14-33bfced7e925 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:32.630238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 211/419] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:32.635667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485116-c71b-4a74-949f-d02e61a0235f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:32.636107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485116-c71b-4a74-949f-d02e61a0235f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:32.636206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485116-c71b-4a74-949f-d02e61a0235f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:32.660238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-54485116-c71b-4a74-949f-d02e61a0235f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:32.661486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0571d-5fcf-475c-aac2-0f414034434f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:32.661775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0571d-5fcf-475c-aac2-0f414034434f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:32.661941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-54485116-c71b-4a74-949f-d02e61a0235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:32.662069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0571d-5fcf-475c-aac2-0f414034434f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:32.663689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 212/420] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:32.669657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce36f-3c2c-4c3e-9f9e-4d61b9e348f1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:32.669947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce36f-3c2c-4c3e-9f9e-4d61b9e348f1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:32.670278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce36f-3c2c-4c3e-9f9e-4d61b9e348f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:32.705579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d0bce36f-3c2c-4c3e-9f9e-4d61b9e348f1 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:32.706800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d0bce36f-3c2c-4c3e-9f9e-4d61b9e348f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:32.732466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:32.796628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 209/421] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:32.817885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 213/422] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:33.152118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54485116-c71b-4a74-949f-d02e61a0235f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:33.164258 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce36f-3c2c-4c3e-9f9e-4d61b9e348f1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:33.196787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:33.304236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce36f-3c2c-4c3e-9f9e-4d61b9e348f1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:33.321591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54485116-c71b-4a74-949f-d02e61a0235f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:33.358372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee01cb-676d-4fd3-a6d1-7ffa9ac7fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:33.359174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee01cb-676d-4fd3-a6d1-7ffa9ac7fa9f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:33.359459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee01cb-676d-4fd3-a6d1-7ffa9ac7fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:33.359728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee01cb-676d-4fd3-a6d1-7ffa9ac7fa9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:33.372042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee01cb-676d-4fd3-a6d1-7ffa9ac7fa9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d9ee01cb-676d-4fd3-a6d1-7ffa9ac7fa9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:33.377840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:33.378764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:33.379611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 210/423] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:33.388881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429f9a5-9c8a-49f3-bdf1-165fa963bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:33.389519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429f9a5-9c8a-49f3-bdf1-165fa963bbba None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:33.389792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429f9a5-9c8a-49f3-bdf1-165fa963bbba None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:33.390058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429f9a5-9c8a-49f3-bdf1-165fa963bbba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:33.391262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee01cb-676d-4fd3-a6d1-7ffa9ac7fa9f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:33.408272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429f9a5-9c8a-49f3-bdf1-165fa963bbba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3429f9a5-9c8a-49f3-bdf1-165fa963bbba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:33.413712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:33.414635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:33.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 214/424] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:33.424531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429f9a5-9c8a-49f3-bdf1-165fa963bbba None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:33.426722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:33.558873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee01cb-676d-4fd3-a6d1-7ffa9ac7fa9f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:33.595191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45adf1-0e05-40b1-accb-23f684911e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:33.595761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:33.595892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45adf1-0e05-40b1-accb-23f684911e69 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:33.596066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45adf1-0e05-40b1-accb-23f684911e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:33.596329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45adf1-0e05-40b1-accb-23f684911e69 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:33.596997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45adf1-0e05-40b1-accb-23f684911e69 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:33.597538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45adf1-0e05-40b1-accb-23f684911e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:33.599489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45adf1-0e05-40b1-accb-23f684911e69 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:33.608272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 211/425] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:33.609611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cdbd-ee22-4276-91fb-535b017ee36f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:33.609791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cdbd-ee22-4276-91fb-535b017ee36f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:33.610041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cdbd-ee22-4276-91fb-535b017ee36f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:33.616065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429f9a5-9c8a-49f3-bdf1-165fa963bbba None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:33.620405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 212/426] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:33.620405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364cdbd-ee22-4276-91fb-535b017ee36f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:33.675093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185e38e-5e3e-4e20-8fa9-5f2a29cf3943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:33.675664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185e38e-5e3e-4e20-8fa9-5f2a29cf3943 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:33.675954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185e38e-5e3e-4e20-8fa9-5f2a29cf3943 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:33.676203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185e38e-5e3e-4e20-8fa9-5f2a29cf3943 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:33.677076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185e38e-5e3e-4e20-8fa9-5f2a29cf3943 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:33.677457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185e38e-5e3e-4e20-8fa9-5f2a29cf3943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:33.678876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185e38e-5e3e-4e20-8fa9-5f2a29cf3943 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:33.688138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 215/427] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:33.689401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983514d7-93df-4fa3-b207-f800e9780413 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:33.689682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983514d7-93df-4fa3-b207-f800e9780413 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:33.689948 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983514d7-93df-4fa3-b207-f800e9780413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:33.709674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-983514d7-93df-4fa3-b207-f800e9780413 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:33.710870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-983514d7-93df-4fa3-b207-f800e9780413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:33.734192 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:33.746033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364cdbd-ee22-4276-91fb-535b017ee36f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:33.784222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:33.785230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:33.785707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:33.786383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:33.787237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 216/428] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:33.788283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983514d7-93df-4fa3-b207-f800e9780413 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:33.798410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:33.800124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:33.817266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:33.818236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:33.819145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:33.819145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:33.819145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:33.821738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 213/429] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:33.830680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:33.918103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:33.951253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983514d7-93df-4fa3-b207-f800e9780413 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.015820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b94a19-b074-4a84-97f6-40f7841e9d3d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.018921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.019544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.019815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.020078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.040664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.042240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:34.069440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:34.070418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:34.071296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:34.071296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:34.071296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:34.071970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 217/430] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:34.084864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.085976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.086659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.086941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.087209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.087857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:34.088158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.089831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.110419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 214/431] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:34.111701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.118232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:34.215572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429ed9-75be-4566-bbac-b83aeddd02a6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.247266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:34.248426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca3b5-7bd0-434c-8877-100cbdb3111b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.253560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.254157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.254472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.255264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.255987 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:34.256283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.257656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.274543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 218/432] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:34.274543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.289537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635a693-b34e-4f64-8b8e-893fb68e80a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.290337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a693-b34e-4f64-8b8e-893fb68e80a0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.290613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a693-b34e-4f64-8b8e-893fb68e80a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.291172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a693-b34e-4f64-8b8e-893fb68e80a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.303708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635a693-b34e-4f64-8b8e-893fb68e80a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b635a693-b34e-4f64-8b8e-893fb68e80a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:34.309436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:34.310361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:34.311196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 215/433] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:34.312336 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f0345-13c1-42ac-81a0-4b222cc95eec None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.312336 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f0345-13c1-42ac-81a0-4b222cc95eec None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.312576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f0345-13c1-42ac-81a0-4b222cc95eec None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.318385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 216/434] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:34.318385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f0345-13c1-42ac-81a0-4b222cc95eec None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.382275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1076c-3a9c-4a46-9524-78c654410bdb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.389628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:34.417041 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.417665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.418168 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.418168 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.425839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.427664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.431516 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f0345-13c1-42ac-81a0-4b222cc95eec None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:34.438930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:34.439830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:34.440755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 219/435] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:34.441380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.475270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93e3d9-dc6d-4e4c-853e-812909e5b879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.475810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93e3d9-dc6d-4e4c-853e-812909e5b879 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.476086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93e3d9-dc6d-4e4c-853e-812909e5b879 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.476359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93e3d9-dc6d-4e4c-853e-812909e5b879 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.483693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93e3d9-dc6d-4e4c-853e-812909e5b879 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.485924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93e3d9-dc6d-4e4c-853e-812909e5b879 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.493070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dc93e3d9-dc6d-4e4c-853e-812909e5b879 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:34.499109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:34.500201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 10:12:34.501335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:34.502787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 217/436] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:34.504029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4695a-42b8-48a8-a8a7-1d43d92b3c00 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.504405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4695a-42b8-48a8-a8a7-1d43d92b3c00 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.504888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4695a-42b8-48a8-a8a7-1d43d92b3c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.523902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-83f4695a-42b8-48a8-a8a7-1d43d92b3c00 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:34.525083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-83f4695a-42b8-48a8-a8a7-1d43d92b3c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:34.555509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75d769-0b5a-4b23-90e9-0f69ab928476 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.591698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.592294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.592573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.592841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.593496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:34.593796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.602687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 218/437] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:34.602687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4695a-42b8-48a8-a8a7-1d43d92b3c00 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.602893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.611981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 220/438] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:34.612811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.655819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:34.703640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4695a-42b8-48a8-a8a7-1d43d92b3c00 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.747336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079a6ec-1ffa-432c-b721-4d8d44d853dd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.748063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.749123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.749584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.750048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.750994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:34.751488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.753095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.763219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:34.773064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 219/439] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:34.773902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.791280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a802d-f61e-477e-9db6-9d5c259f8bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.792460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a802d-f61e-477e-9db6-9d5c259f8bbc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.792885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a802d-f61e-477e-9db6-9d5c259f8bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.793158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a802d-f61e-477e-9db6-9d5c259f8bbc None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.793926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a802d-f61e-477e-9db6-9d5c259f8bbc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:34.794230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a802d-f61e-477e-9db6-9d5c259f8bbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.795842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a802d-f61e-477e-9db6-9d5c259f8bbc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.819144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 221/440] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:34.819144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216892-3b13-4626-8f1d-289fee4bb981 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.819144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216892-3b13-4626-8f1d-289fee4bb981 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.819144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216892-3b13-4626-8f1d-289fee4bb981 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.822206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 222/441] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:34.823032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216892-3b13-4626-8f1d-289fee4bb981 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.886353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfbc5c-2b38-4f8f-bd8c-a57661da0277 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.902296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:34.925102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.925718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.925992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.926258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:34.931021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216892-3b13-4626-8f1d-289fee4bb981 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:34.936823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:34.938242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:34.952452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:34.953496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:34.954989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:34.954989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:34.954989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:34.954989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 220/442] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:34.954989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:34.967892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59b7e-bae4-4a39-8057-0238bdf19897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:34.968513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59b7e-bae4-4a39-8057-0238bdf19897 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:34.968792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59b7e-bae4-4a39-8057-0238bdf19897 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:34.969065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59b7e-bae4-4a39-8057-0238bdf19897 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.012705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.042688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59b7e-bae4-4a39-8057-0238bdf19897 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.044309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59b7e-bae4-4a39-8057-0238bdf19897 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.053986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471c2c3-797b-46d6-82bf-eb41f59af9ad None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.087392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be595d50-15ea-42e4-9d6f-838480de623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.088016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be595d50-15ea-42e4-9d6f-838480de623b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.088292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be595d50-15ea-42e4-9d6f-838480de623b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.088562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be595d50-15ea-42e4-9d6f-838480de623b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.100397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be595d50-15ea-42e4-9d6f-838480de623b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.101829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be595d50-15ea-42e4-9d6f-838480de623b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-be595d50-15ea-42e4-9d6f-838480de623b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:35.116092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:35.117187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:35.118371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:35.118371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:35.118371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:35.118371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 221/443] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:35.118632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127559a3-4641-4551-a217-83c5a3f31e0a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.118870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127559a3-4641-4551-a217-83c5a3f31e0a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.119138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127559a3-4641-4551-a217-83c5a3f31e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.136423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-127559a3-4641-4551-a217-83c5a3f31e0a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:35.137607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-127559a3-4641-4551-a217-83c5a3f31e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:35.148358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.148529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 223/444] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:12:35.153908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59b7e-bae4-4a39-8057-0238bdf19897 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.205154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 222/445] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:35.205997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127559a3-4641-4551-a217-83c5a3f31e0a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.235529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.268634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59b7e-bae4-4a39-8057-0238bdf19897 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.306239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a97e8-ba9d-443e-a1d9-10c675497fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.306898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a97e8-ba9d-443e-a1d9-10c675497fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.307172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a97e8-ba9d-443e-a1d9-10c675497fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.307437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a97e8-ba9d-443e-a1d9-10c675497fc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.309914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127559a3-4641-4551-a217-83c5a3f31e0a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.317955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a97e8-ba9d-443e-a1d9-10c675497fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-452a97e8-ba9d-443e-a1d9-10c675497fc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:35.323183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:35.324337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 224/446] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:35.325150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a97e8-ba9d-443e-a1d9-10c675497fc6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.352445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.353191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.353558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.353920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.354681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:35.355101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.356607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.371089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 223/447] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:35.375278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.384959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.434146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a97e8-ba9d-443e-a1d9-10c675497fc6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.461614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.468256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.469014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.469454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.469812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.470601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:35.470901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.472565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.484408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 225/448] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:35.485190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.485538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f77e06-ea20-4e4c-bc6e-971f487da092 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.522216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.522794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.523070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.523343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.533913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.535386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:35.549447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:35.550529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:35.552112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:35.552112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:35.552112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:35.552112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 224/449] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:35.552622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.590055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b598e0-ba5a-43ef-9de5-6599fc373708 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.608794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.623691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.624368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.624645 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.624913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.625571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:35.625870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.627269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.635056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 226/450] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:35.635832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.649893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ca31a-e324-4b92-8efe-d4b9167fe9da None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.684629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.685331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.685678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.686014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.686633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.694620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.696126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:35.707042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:35.708721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:35.709759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 225/451] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:35.710354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.736951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d46771-a652-4c69-91f2-94e4467b53d8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.769327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae9e87-3af2-4870-a6f9-109168d50585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.769968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae9e87-3af2-4870-a6f9-109168d50585 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.770798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae9e87-3af2-4870-a6f9-109168d50585 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.771084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae9e87-3af2-4870-a6f9-109168d50585 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.781016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae9e87-3af2-4870-a6f9-109168d50585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-66ae9e87-3af2-4870-a6f9-109168d50585 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:35.786130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:35.787185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:35.787989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:35.790469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 227/452] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:35.791474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae9e87-3af2-4870-a6f9-109168d50585 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.811390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63217b-8699-4ab8-a3bf-651a967beceb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.828492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.846114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.846761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.847035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.847549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.854753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.856191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:35.868240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:35.869291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 226/453] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:35.870553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.896203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae9e87-3af2-4870-a6f9-109168d50585 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:35.908735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:35.929651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:35.930241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:35.930541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:35.930809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:35.931435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:35.931732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:35.933107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:35.944738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 228/454] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:35.945513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:35.974233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87a63-38e0-48fe-826f-973a5f2667e2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.008008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.008622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.008893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.009158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.019263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.021284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:36.034865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:36.037043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:36.038106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:36.038106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:36.038106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:36.038106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 227/455] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:36.038106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.047939 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d19705-f500-4f7f-9bbd-f7e6eae43146 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.070729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.085052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.085654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.085931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.086198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.086970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:36.087270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.088674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.106114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 229/456] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Oct 17 10:12:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:36.107073 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.135773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.145962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb964e-8f31-427d-b2d1-3c5a1b6667e9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.179596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.180176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.180451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.180712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.181366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:36.181664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.183041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.190832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 228/457] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:36.196386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.205213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8cef9-54a2-4711-9115-95809c1e15c9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.238650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.239753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.240106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.240385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.241068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:36.241371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.242849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.254291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 230/458] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:36.255245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.284954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.294115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b50d89-b383-45fb-a5bc-3ee798306917 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.328358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.328977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.329637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.329903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.337204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.338637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.344803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:36.350296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:36.351352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 229/459] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:36.352223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.358295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f1ab9-b3f7-4811-af4c-0acadba44a17 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.391336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.391927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.392202 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.392469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.393107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:36.393387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.394788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.406157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 231/460] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:36.406962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.454860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5cdb-5644-42d9-a78f-744b4bfe27dd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.488919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2931a7-0812-4630-8976-23161d3c715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.489507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2931a7-0812-4630-8976-23161d3c715f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.489782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2931a7-0812-4630-8976-23161d3c715f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.490051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2931a7-0812-4630-8976-23161d3c715f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.503605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc886-f2c2-4a55-a4bd-0db238634aec None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.513063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.539221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.539772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.540051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.540322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.550076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.551572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.559642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2931a7-0812-4630-8976-23161d3c715f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.561092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2931a7-0812-4630-8976-23161d3c715f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:36.565944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:36.567566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:36.568633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:36.568633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:36.568633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:36.572535 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 232/461] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:36.573337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.576141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.658398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 230/462] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:12:36.659640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2931a7-0812-4630-8976-23161d3c715f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.674382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6444c7-f0ba-47a1-abdc-388ac8cb400e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.708705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c88c8-6565-47c3-a690-7069b03cb9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.709356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c88c8-6565-47c3-a690-7069b03cb9e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.709629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c88c8-6565-47c3-a690-7069b03cb9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.709894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c88c8-6565-47c3-a690-7069b03cb9e8 None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/cf85319814ff4478b40b8a2e28e054ed/roles/4045fa40bdd54e9f889b47e1fb2704f5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.711783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.724743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c88c8-6565-47c3-a690-7069b03cb9e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=cf85319814ff4478b40b8a2e28e054ed, role_id=4045fa40bdd54e9f889b47e1fb2704f5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.726179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c88c8-6565-47c3-a690-7069b03cb9e8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.741129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 233/463] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:12:36 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/cf85319814ff4478b40b8a2e28e054ed/roles/4045fa40bdd54e9f889b47e1fb2704f5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:36.741925 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c88c8-6565-47c3-a690-7069b03cb9e8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.776924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2931a7-0812-4630-8976-23161d3c715f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.806003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.820373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.821004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.821278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.821545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.822250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:36.823509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.824923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.838982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 231/464] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:36.839750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.869396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c88c8-6565-47c3-a690-7069b03cb9e8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.907008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.907653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.907924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.908185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.917157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.919103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:36.932003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:36.932967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:36.933886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 234/465] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:36.934793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:36.954684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c523d-79fa-482a-973d-32dabc368898 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:36.970016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:36.993509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:36.994110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:36.994412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:36.994684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:36.995326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:36.995702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:36.997277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.010958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 232/466] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:37.012362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.050432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621f44f-bd7d-4568-b6a1-211888e15d54 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.063566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:37.090746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.091381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.091658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.092100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:37.105701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:37.107193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:37.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:37.122580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:37.123686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:37.123686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:37.123686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:37.127672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 235/467] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:37.128296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.133310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f79ebe-19d4-4985-83e5-bda777c7383c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.171671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.172267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.172534 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.173107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:37.173828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] RBAC: Adding query filter params (role.id=4045fa40bdd54e9f889b47e1fb2704f5, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=9d52624718de4457adc44e9965cd7a09) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:37.174130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:37.175583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.186368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 233/468] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/role_assignments?user.id=9d52624718de4457adc44e9965cd7a09&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=4045fa40bdd54e9f889b47e1fb2704f5 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:37.186368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.237952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c11bc7-52d5-487b-bc8f-245edb1d26ce None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.241600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:37.294390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.295633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.295909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.296177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:37.297202 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:37.297502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:37.300223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.327784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 236/469] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:37.328585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.339701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afa898-47ec-4483-bb19-e2457bcfb89a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.364491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:37.381473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.381694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.381959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.382221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:37.392085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:37.393536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:37.409766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:37.411291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 10:12:37.412217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:37.416650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 234/470] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:37.416928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.463117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2e81e-e953-4b56-a8b9-e425288ef486 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.498843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.499449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.499725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.499992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:37.500628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:37.500926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:37.502365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.514577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 237/471] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:37.519240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.527979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:37.537943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e064d-97f4-4880-b383-33c9e74c8c13 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.580530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d649d8f-d62b-4c05-835d-8bbb5328e0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.581249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d649d8f-d62b-4c05-835d-8bbb5328e0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.581523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d649d8f-d62b-4c05-835d-8bbb5328e0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.581887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d649d8f-d62b-4c05-835d-8bbb5328e0b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:37.610298 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:37.637206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366e778-0d4a-4bcd-8faa-d19be13aa3df None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.665827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d649d8f-d62b-4c05-835d-8bbb5328e0b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:37.667583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d649d8f-d62b-4c05-835d-8bbb5328e0b9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.686289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.687502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.687947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.688353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:37.689213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:37.689651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:37.692821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.723964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 238/472] 10.222.0.16 () {56 vars in 1139 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:37.725079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.777644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:37.786897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 235/473] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:12:37.790202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d649d8f-d62b-4c05-835d-8bbb5328e0b9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.855181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1c883-3c76-46ad-93d2-2ad3a22be798 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.890672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.891549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.891863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.892229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:37.899318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:37.900880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:37.903918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:37.915246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:37.916274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:37.917310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:37.917310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:37.917310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:37.917310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:37.917310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:37.917310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:37.917310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:37.917310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:37.921715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d649d8f-d62b-4c05-835d-8bbb5328e0b9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:37.923982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 239/474] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:37.925690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:37.967743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ef0dc-721c-4ace-a015-ad76e4e44454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:37.968340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ef0dc-721c-4ace-a015-ad76e4e44454 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:37.968610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ef0dc-721c-4ace-a015-ad76e4e44454 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:37.968875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ef0dc-721c-4ace-a015-ad76e4e44454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.036906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68570c-397c-4c34-9574-68213f3145ef None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.049173 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:38.059587 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ef0dc-721c-4ace-a015-ad76e4e44454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.061770 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ef0dc-721c-4ace-a015-ad76e4e44454 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.086647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.087279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.087554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.087823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.102564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.104026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:38.118619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:38.119499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:38.120332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:38.120332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:38.120332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:38.120332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 240/475] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:38.120543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.175463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:38.204805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 236/476] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 10:12:38.205819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ef0dc-721c-4ace-a015-ad76e4e44454 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.244833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed982b-4268-4b1d-9cdf-2fe31e4cdab0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.288659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.290074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.290638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.291046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.291892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:38.292581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.294940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.307295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:38.314158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 241/477] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:38.315163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.315457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ef0dc-721c-4ace-a015-ad76e4e44454 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.366358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.367381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.367663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.367931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.368911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:38.369155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.370669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.384266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 237/478] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:38.390082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.434559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8eaf-0b17-4adf-b48c-1c376b3cb180 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.449431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:38.479270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.480359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.480637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.480901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.496048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.497525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.508860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56691-8ebc-4987-96bb-8d9af9022f81 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:38.513952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:38.514896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:38.515760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:38.515760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:38.515760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:38.515760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 242/479] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:38.516506 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.553899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.554533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.554808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.555167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.555978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] RBAC: Adding query filter params (role.id=4045fa40bdd54e9f889b47e1fb2704f5, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=24f1a6c05ed94f5fa81dd355497b6754) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:38.556284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.557689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.564204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 238/480] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/role_assignments?user.id=24f1a6c05ed94f5fa81dd355497b6754&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=4045fa40bdd54e9f889b47e1fb2704f5 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:38.565140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.590710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:38.667295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef2b41-9b59-459c-97a9-2191ba783071 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.680844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7b59c-aa0b-4d9f-ba41-47b0e45ee3e4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.719866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.720844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.721289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.721758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.723079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:38.724117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.725633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.734908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.735545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.735822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.736094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.740136 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 239/481] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:38.748762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.748762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.755298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:38.766032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:38.767142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:38.768120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 243/482] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:38.773822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.809182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:38.862542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:38.887160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3cd31-03ce-4a58-bcf3-1f73d9aae62f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.937645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.938992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1777897-1919-4b4c-a1a1-27c80386354d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:38.939380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.939799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.940221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.952233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.954018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:38.969767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:38.970765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:38.971977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 240/483] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:38.978486 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:38.988778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:38.989688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:38.990131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:38.990592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:38.991883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:38.992744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:38.994649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:39.017457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 244/484] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:39.023724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:39.104908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7947835-bb37-4adb-b82f-11b29b8173cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:39.142275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7def7b-f7df-4709-9d0b-2d22e09b0b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:39.142902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7def7b-f7df-4709-9d0b-2d22e09b0b22 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:39.143176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7def7b-f7df-4709-9d0b-2d22e09b0b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:39.143441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7def7b-f7df-4709-9d0b-2d22e09b0b22 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:39.144244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7def7b-f7df-4709-9d0b-2d22e09b0b22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:39.144543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7def7b-f7df-4709-9d0b-2d22e09b0b22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:39.146891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7def7b-f7df-4709-9d0b-2d22e09b0b22 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:39.156653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:39.162880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 241/485] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:39.162880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739c5db-27ef-46f0-a978-c604c9bda6da None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:39.162880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739c5db-27ef-46f0-a978-c604c9bda6da None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:39.162880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739c5db-27ef-46f0-a978-c604c9bda6da None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:39.163938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 242/486] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:39.168997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739c5db-27ef-46f0-a978-c604c9bda6da None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:39.179678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:39.199020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3f8e7-b172-46a7-b7d1-ba0ae3bbad27 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:39.236081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d89a5-18b3-4909-a507-c9669975f1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:39.236654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d89a5-18b3-4909-a507-c9669975f1da None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:39.236973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d89a5-18b3-4909-a507-c9669975f1da None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:39.237239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d89a5-18b3-4909-a507-c9669975f1da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:39.238013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d89a5-18b3-4909-a507-c9669975f1da None admin] RBAC: Adding query filter params (role.id=4045fa40bdd54e9f889b47e1fb2704f5, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=9ac934ed0122470cb489d6b324c664f6) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:39.238338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d89a5-18b3-4909-a507-c9669975f1da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:39.240432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d89a5-18b3-4909-a507-c9669975f1da None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:39.246805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 245/487] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:39 2022] GET /identity/v3/role_assignments?user.id=9ac934ed0122470cb489d6b324c664f6&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=4045fa40bdd54e9f889b47e1fb2704f5 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:39.247983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88bfdf-bd8a-4f83-93ff-53316ca80777 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:39.248262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88bfdf-bd8a-4f83-93ff-53316ca80777 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:39.248534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88bfdf-bd8a-4f83-93ff-53316ca80777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:39.268381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6a88bfdf-bd8a-4f83-93ff-53316ca80777 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:39.269369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6a88bfdf-bd8a-4f83-93ff-53316ca80777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:39.311123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739c5db-27ef-46f0-a978-c604c9bda6da None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:39.357138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 246/488] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:39.359081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3356d-fe58-4fd7-bb7c-cc4d1bf45151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:39.359951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3356d-fe58-4fd7-bb7c-cc4d1bf45151 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:39.360427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3356d-fe58-4fd7-bb7c-cc4d1bf45151 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:39.360910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3356d-fe58-4fd7-bb7c-cc4d1bf45151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:39.361851 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3356d-fe58-4fd7-bb7c-cc4d1bf45151 None admin] RBAC: Adding query filter params (role.id=4045fa40bdd54e9f889b47e1fb2704f5, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=bd7883e3f6ab4f58969f51ffd439b1c0) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:39.362381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3356d-fe58-4fd7-bb7c-cc4d1bf45151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:39.364671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3356d-fe58-4fd7-bb7c-cc4d1bf45151 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:39.372830 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 243/489] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:39 2022] GET /identity/v3/role_assignments?user.id=bd7883e3f6ab4f58969f51ffd439b1c0&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=4045fa40bdd54e9f889b47e1fb2704f5 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:39.516162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2e6b-555a-40d8-ae17-1b1c8296cfd8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:39.516528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2e6b-555a-40d8-ae17-1b1c8296cfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:39.516718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2e6b-555a-40d8-ae17-1b1c8296cfd8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:39.518289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 247/490] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:39.524059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1617d-cded-43ef-aec4-23a26b9d959c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:39.524339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1617d-cded-43ef-aec4-23a26b9d959c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:39.524610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1617d-cded-43ef-aec4-23a26b9d959c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:39.544698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bdd1617d-cded-43ef-aec4-23a26b9d959c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:39.545919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bdd1617d-cded-43ef-aec4-23a26b9d959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:39.619059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:39.629295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 244/491] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:39.676545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e2e6b-555a-40d8-ae17-1b1c8296cfd8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:39.769533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:39.796994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e2e6b-555a-40d8-ae17-1b1c8296cfd8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:39.843195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e317265-822f-4b67-9610-c0ed7eaf0626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:39.843829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e317265-822f-4b67-9610-c0ed7eaf0626 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:39.844106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e317265-822f-4b67-9610-c0ed7eaf0626 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:39.844381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e317265-822f-4b67-9610-c0ed7eaf0626 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:39.863120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e317265-822f-4b67-9610-c0ed7eaf0626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5e317265-822f-4b67-9610-c0ed7eaf0626 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:39.868390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:39.869266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:39.870051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 248/492] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:39.875433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1617d-cded-43ef-aec4-23a26b9d959c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:39.910118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e317265-822f-4b67-9610-c0ed7eaf0626 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:40.043735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1617d-cded-43ef-aec4-23a26b9d959c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:40.106514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e317265-822f-4b67-9610-c0ed7eaf0626 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:40.119165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:40.119828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:40.120150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:40.120423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:40.121154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:40.121454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:40.122914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:40.139665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 245/493] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:40.144576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:40.179050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49620ee-3828-47f0-b8fc-de877ecebc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:40.179732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49620ee-3828-47f0-b8fc-de877ecebc0e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:40.180013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49620ee-3828-47f0-b8fc-de877ecebc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:40.180280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49620ee-3828-47f0-b8fc-de877ecebc0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:40.190589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:40.197314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49620ee-3828-47f0-b8fc-de877ecebc0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a49620ee-3828-47f0-b8fc-de877ecebc0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:40.202284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:40.203670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:40.204637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 249/494] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:40.214440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49620ee-3828-47f0-b8fc-de877ecebc0e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:40.259531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:40.360547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f37d6-36bb-4979-8fc6-669d244acfcc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:40.363125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49620ee-3828-47f0-b8fc-de877ecebc0e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:40.434668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:40.435586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:40.436369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:40.437136 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:40.437798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:40.438028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:40.438702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:40.439062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:40.439872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:40.440832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:40.443349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:40.455968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:40.457876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:40.465244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 250/495] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:40.470461 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:40.478097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:40.479093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:40.480174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:40.480174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:40.480174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:40.482249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 246/496] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:40.487429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:40.588990 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:40.658971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb908-d74e-4d26-b347-fb85fca7ef6c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:40.669713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:40.683931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8942-1e29-49f9-8421-2ca76a23e26c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:40.701867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:40.703159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:40.703434 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:40.703702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:40.704351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:40.704650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:40.706010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:40.721853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 247/497] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:40.728247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:40.749485 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:40.750847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:40.751124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:40.751395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:40.771799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:40.773252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:40.804058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:40.805054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:40.806016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:40.806016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:40.806016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:40.807969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 251/498] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:40.813440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:40.916947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063489a1-d21d-431b-9aea-79a8a6d2767f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:40.959732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:40.960353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:40.960627 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:40.960890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:40.968233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:40.969682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:40.982683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:40.983659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:40.984595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 248/499] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:40.990917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:40.997758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02397b74-fe04-4c00-916a-9fb80c7dc2c6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:41.029167 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:41.046003 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:41.086355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:41.086355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.086355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.086355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.086731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:41.086731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:41.087139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:41.106270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 252/500] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:41.113158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:41.191342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98789f94-3b19-4e85-b164-eec7333f3526 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:41.264420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:41.265022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.265295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.265564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.265758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b327e-f6f7-47fc-bf55-6c5988348907 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:41.266287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:41.266611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:41.268032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:41.306591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 249/501] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Oct 17 10:12:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:41.314165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:41.314828 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.315105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.315375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.317868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:41.335239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:41.336696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:41.354993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:41.357284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:41.359510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 253/502] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Oct 17 10:12:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:41.364925 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:41.406453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:41.473780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:41.502830 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30601851-799d-471c-b052-8491c8d4f1f2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:41.543857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468ef6c-bffe-446f-a512-d770f20f53ad None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:41.572292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a9b66-0c06-444d-a0eb-1a0b22e294ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:41.572909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a9b66-0c06-444d-a0eb-1a0b22e294ed None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.573186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a9b66-0c06-444d-a0eb-1a0b22e294ed None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.573454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a9b66-0c06-444d-a0eb-1a0b22e294ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.612599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3070d-bb5e-4b11-affe-16317bcb1293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:41.613265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3070d-bb5e-4b11-affe-16317bcb1293 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.613542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3070d-bb5e-4b11-affe-16317bcb1293 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.614397 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3070d-bb5e-4b11-affe-16317bcb1293 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.615239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3070d-bb5e-4b11-affe-16317bcb1293 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:41.615595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3070d-bb5e-4b11-affe-16317bcb1293 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:41.617140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3070d-bb5e-4b11-affe-16317bcb1293 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:41.641343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 254/503] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Oct 17 10:12:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:41.642802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eef97-4439-4843-b3b5-d3b6802d673b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.643080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eef97-4439-4843-b3b5-d3b6802d673b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.643444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eef97-4439-4843-b3b5-d3b6802d673b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.645043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 255/504] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:41.646141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779eef97-4439-4843-b3b5-d3b6802d673b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:41.667202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a9b66-0c06-444d-a0eb-1a0b22e294ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:41.668711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a9b66-0c06-444d-a0eb-1a0b22e294ed None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:41.751635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:41.772039 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779eef97-4439-4843-b3b5-d3b6802d673b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:41.774096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:41.792604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 250/505] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 10:12:41.794123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008232-d865-4e61-8ae6-4f1e30172361 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.794685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008232-d865-4e61-8ae6-4f1e30172361 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.795171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008232-d865-4e61-8ae6-4f1e30172361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.813819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dc008232-d865-4e61-8ae6-4f1e30172361 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:41.815413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc008232-d865-4e61-8ae6-4f1e30172361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:41.821410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:41.822041 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.822316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.822802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.833830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:41.835446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:41.851704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:41.852701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:41.853597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:41.853597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:41.853597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:41.853597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 256/506] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:41.853845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:41.898268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 251/507] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:41.899555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2100f-bad8-4f71-bae7-7df178ac7b9b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:41.899988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2100f-bad8-4f71-bae7-7df178ac7b9b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:41.900262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2100f-bad8-4f71-bae7-7df178ac7b9b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:41.905537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 252/508] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:41.905537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2100f-bad8-4f71-bae7-7df178ac7b9b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:41.975518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d0bcb-8db8-4200-8e2f-0bffdd62a0c1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.021746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2100f-bad8-4f71-bae7-7df178ac7b9b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.022561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a20bdc-d058-473d-b7bb-f523ac805738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.023403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a20bdc-d058-473d-b7bb-f523ac805738 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.023688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a20bdc-d058-473d-b7bb-f523ac805738 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.023959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a20bdc-d058-473d-b7bb-f523ac805738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.039239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a20bdc-d058-473d-b7bb-f523ac805738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.040975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a20bdc-d058-473d-b7bb-f523ac805738 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.041860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:42.043614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:42.062732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cb945-95d0-47c3-890c-44efebbacf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.063379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cb945-95d0-47c3-890c-44efebbacf46 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.063645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cb945-95d0-47c3-890c-44efebbacf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.063914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cb945-95d0-47c3-890c-44efebbacf46 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.064559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cb945-95d0-47c3-890c-44efebbacf46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:42.064856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cb945-95d0-47c3-890c-44efebbacf46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-53a20bdc-d058-473d-b7bb-f523ac805738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:42.066152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:42.067284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:42.068335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:42.068335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:42.068335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:42.068538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cb945-95d0-47c3-890c-44efebbacf46 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.072802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 257/509] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:42.072802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7acaf-4d04-424d-8433-1e89e68589a9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.072802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7acaf-4d04-424d-8433-1e89e68589a9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.072802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7acaf-4d04-424d-8433-1e89e68589a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.086628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 253/510] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:42.087722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411459e-5dab-4873-aa0a-e7d3e2154780 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.088002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411459e-5dab-4873-aa0a-e7d3e2154780 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.088267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411459e-5dab-4873-aa0a-e7d3e2154780 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.093642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 254/511] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:42.093642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411459e-5dab-4873-aa0a-e7d3e2154780 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.100008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e3d7acaf-4d04-424d-8433-1e89e68589a9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:42.101198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e3d7acaf-4d04-424d-8433-1e89e68589a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:42.173469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 258/512] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:42.174354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7acaf-4d04-424d-8433-1e89e68589a9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.212685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411459e-5dab-4873-aa0a-e7d3e2154780 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.247838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad62e1a-40c0-4b75-9fe8-37eb8e709a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.248436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad62e1a-40c0-4b75-9fe8-37eb8e709a69 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.248713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad62e1a-40c0-4b75-9fe8-37eb8e709a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.248984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad62e1a-40c0-4b75-9fe8-37eb8e709a69 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.249618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad62e1a-40c0-4b75-9fe8-37eb8e709a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:42.249920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad62e1a-40c0-4b75-9fe8-37eb8e709a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.251321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad62e1a-40c0-4b75-9fe8-37eb8e709a69 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.262879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 255/513] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:42.263879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e739d87-f0e3-490c-9328-2fcb11351c6c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.264158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e739d87-f0e3-490c-9328-2fcb11351c6c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.264425 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e739d87-f0e3-490c-9328-2fcb11351c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.291950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5e739d87-f0e3-490c-9328-2fcb11351c6c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:42.293050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e739d87-f0e3-490c-9328-2fcb11351c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:42.299125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7acaf-4d04-424d-8433-1e89e68589a9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.322972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:42.327541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:42.334916 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.336338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.336630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.336903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.344777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.346240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:42.359118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:42.360019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 259/514] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:42.360914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.378664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 256/515] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:42.378664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e739d87-f0e3-490c-9328-2fcb11351c6c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.491086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e739d87-f0e3-490c-9328-2fcb11351c6c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.500121 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52bd7-7daa-413a-8250-2fe9fe0d5d92 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.525796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.526450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.526724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.526991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.533541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.534987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.536603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.537482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.537978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.538147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.538756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.540115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:42.546454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:42.547316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:42.548250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 257/516] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:42.548746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:42.556028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:42.557034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:42.557898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 260/517] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:42.560256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.581028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:42.613686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:42.656138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8610fb-b2ec-4517-9ece-13728fc225e9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.668091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c014220-6b68-48ac-ab10-d7a35aa003c3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.689918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.690625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.690899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.691159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.691840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:42.692303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.693739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.702570 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.703183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.703453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.703717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] PATH_INFO: `/v3/services/network` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.704284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.705629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.705754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 258/518] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:42.706440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:42.720293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:42.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:42.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 261/519] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:42.723978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.796661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:42.812101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fc729-1073-44ad-a0b2-599a77cef9cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.823241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c186-5f19-4900-afe5-8b4dbce716bb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.836476 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:42.845417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.846011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.846290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.846587 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.847267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:42.847828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.849345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.859096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.859699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.859978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.860250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:42.860529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 259/520] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:42.860874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:42.861182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:42.861335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.862599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:42.874945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 262/521] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:42.874945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:42.962268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35492b16-1b76-4fc2-b263-293bd74c209d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.970169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f60cb-e9e8-4c18-bf0c-118290a4c7b6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:42.995602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:42.996217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:42.996484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:42.996749 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:43.003020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28624a-afd3-4f8c-b18f-dd3497cc9839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:43.003663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28624a-afd3-4f8c-b18f-dd3497cc9839 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:43.003937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28624a-afd3-4f8c-b18f-dd3497cc9839 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:43.004213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28624a-afd3-4f8c-b18f-dd3497cc9839 None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/9d52624718de4457adc44e9965cd7a09/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:43.006338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28624a-afd3-4f8c-b18f-dd3497cc9839 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=9d52624718de4457adc44e9965cd7a09, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:43.007317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:43.007799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28624a-afd3-4f8c-b18f-dd3497cc9839 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:43.008712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:43.015603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:43.022521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 263/522] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:12:42 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/9d52624718de4457adc44e9965cd7a09/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:43.023309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28624a-afd3-4f8c-b18f-dd3497cc9839 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:43.024862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:43.025994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:43.027165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:43.027165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:43.027165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:43.027165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 260/523] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:43.027165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:43.085124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:43.154254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61bdd1-7178-41e4-a6b3-db1524e40adc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:43.168244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28624a-afd3-4f8c-b18f-dd3497cc9839 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:43.195053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:43.195656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:43.195934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:43.196198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:43.196918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] RBAC: Adding query filter params (role.id=4045fa40bdd54e9f889b47e1fb2704f5, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=cf85319814ff4478b40b8a2e28e054ed) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:43.197219 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:43.199452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:43.206223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 261/524] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:43 2022] GET /identity/v3/role_assignments?user.id=cf85319814ff4478b40b8a2e28e054ed&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=4045fa40bdd54e9f889b47e1fb2704f5 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:43.207039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:43.212411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:43.213080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:43.213299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:43.213566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:43.214204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] RBAC: Adding query filter params (name=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:43.214537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:43.215950 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:43.228193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 264/525] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:12:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:43.228193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:43.303212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:43.354228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:43.354608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe438-f0a4-4198-8680-0b22de342654 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:43.357502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e163ba-50c1-4457-8697-7dd984d72368 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:43.398550 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:43.399558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:43.399836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:43.400090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:43.400732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:43.401034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:43.401573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:43.402174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:43.402424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:43.402522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:43.402738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:43.403372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:43.403664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:43.405008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:43.408749 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 262/526] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Oct 17 10:12:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:43.409554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:43.428559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 265/527] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:43.434421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:43.537539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83c43-ca4f-4fa5-bd7e-c73f8144962c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:43.580589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:43.588022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f064dea-99f2-442b-a493-df4610da8381 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:43.602864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac94303-2744-4e55-820d-906b6fe2a11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:43.603485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac94303-2744-4e55-820d-906b6fe2a11d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:43.603758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac94303-2744-4e55-820d-906b6fe2a11d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:43.604025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac94303-2744-4e55-820d-906b6fe2a11d None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:43.604729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac94303-2744-4e55-820d-906b6fe2a11d None admin] RBAC: Adding query filter params (type=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:43.604968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac94303-2744-4e55-820d-906b6fe2a11d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:43.606392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac94303-2744-4e55-820d-906b6fe2a11d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:43.616414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 263/528] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:12:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:43.644667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9d60d-c9b9-4ca4-bfac-1201bd6dee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:43.645833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9d60d-c9b9-4ca4-bfac-1201bd6dee8f None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:43.646104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9d60d-c9b9-4ca4-bfac-1201bd6dee8f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:43.646394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9d60d-c9b9-4ca4-bfac-1201bd6dee8f None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:43.647061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9d60d-c9b9-4ca4-bfac-1201bd6dee8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:43.648568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9d60d-c9b9-4ca4-bfac-1201bd6dee8f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:43.683452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 266/529] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:12:43 2022] POST /identity/v3/users => generated 254 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:45.689478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37917141-6ea9-4d8a-8b66-e84d0f20c9f1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:45.690519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37917141-6ea9-4d8a-8b66-e84d0f20c9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:45.691020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37917141-6ea9-4d8a-8b66-e84d0f20c9f1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:45.693509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 264/530] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:45.706021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39540208-0c93-4fa6-b1e3-6e53bdec42d7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:45.706499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39540208-0c93-4fa6-b1e3-6e53bdec42d7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:45.706580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39540208-0c93-4fa6-b1e3-6e53bdec42d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:45.739761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-39540208-0c93-4fa6-b1e3-6e53bdec42d7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:45.740963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-39540208-0c93-4fa6-b1e3-6e53bdec42d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:45.752547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:45.826071 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 267/531] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:46.087386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee460f90-139c-473f-ad40-13013c48deb5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.088016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee460f90-139c-473f-ad40-13013c48deb5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.088508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee460f90-139c-473f-ad40-13013c48deb5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.090493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 265/532] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:46.095996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28165cc-9394-419c-9161-a478da3194e8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.096279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28165cc-9394-419c-9161-a478da3194e8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.096544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28165cc-9394-419c-9161-a478da3194e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.127520 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a28165cc-9394-419c-9161-a478da3194e8 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:46.128720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a28165cc-9394-419c-9161-a478da3194e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:46.175179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee460f90-139c-473f-ad40-13013c48deb5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:46.212210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 268/533] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:46.213801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8423ee7-9e1c-4760-8d94-f77c0b8fc378 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.214246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8423ee7-9e1c-4760-8d94-f77c0b8fc378 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.214736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8423ee7-9e1c-4760-8d94-f77c0b8fc378 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.216369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 269/534] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:46.221782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffbc62-854d-4095-b822-03b7e0d224c8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.222269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffbc62-854d-4095-b822-03b7e0d224c8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.223002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffbc62-854d-4095-b822-03b7e0d224c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.242175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0cffbc62-854d-4095-b822-03b7e0d224c8 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:46.243645 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0cffbc62-854d-4095-b822-03b7e0d224c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:46.316244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee460f90-139c-473f-ad40-13013c48deb5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:46.330915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 270/535] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:46.330915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a825c5-2ebe-40a8-9e53-493bf102de37 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.330915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a825c5-2ebe-40a8-9e53-493bf102de37 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.330915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a825c5-2ebe-40a8-9e53-493bf102de37 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.330915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 271/536] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:46.335517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a084cfc-e008-47f9-bf6b-b927da9cca32 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.336065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a084cfc-e008-47f9-bf6b-b927da9cca32 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.336176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a084cfc-e008-47f9-bf6b-b927da9cca32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.349749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:46.356849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1a084cfc-e008-47f9-bf6b-b927da9cca32 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:46.358033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1a084cfc-e008-47f9-bf6b-b927da9cca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:46.419058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0194117-9f2a-4d00-b1fa-ae403ccac1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:46.420231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0194117-9f2a-4d00-b1fa-ae403ccac1cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.421037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0194117-9f2a-4d00-b1fa-ae403ccac1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.421518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0194117-9f2a-4d00-b1fa-ae403ccac1cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.436114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 272/537] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 10:12:46.442765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0194117-9f2a-4d00-b1fa-ae403ccac1cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c0194117-9f2a-4d00-b1fa-ae403ccac1cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:46.448358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:46.449385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:46.450230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 266/538] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:46.456133 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a084cfc-e008-47f9-bf6b-b927da9cca32 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:46.552102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105b81-c8a2-4866-b1a7-4db12994931f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.552446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105b81-c8a2-4866-b1a7-4db12994931f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.552721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105b81-c8a2-4866-b1a7-4db12994931f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.554043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 267/539] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:12:46.558850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7606c27-6c31-4cec-97e7-ed602eb29200 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.559126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7606c27-6c31-4cec-97e7-ed602eb29200 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.559396 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7606c27-6c31-4cec-97e7-ed602eb29200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.575218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a084cfc-e008-47f9-bf6b-b927da9cca32 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:46.580173 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d7606c27-6c31-4cec-97e7-ed602eb29200 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:46.581526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d7606c27-6c31-4cec-97e7-ed602eb29200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:46.617692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:46.618296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.618646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.618914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.619722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:46.620021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:46.621453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:46.623869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:46.632059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 273/540] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:46.633071 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:46.638089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:46.660353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 268/541] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 10:12:46.661709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7606c27-6c31-4cec-97e7-ed602eb29200 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:46.769274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7606c27-6c31-4cec-97e7-ed602eb29200 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:46.775882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1145b-a2d2-47bf-b465-11ed3c0ef34d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:46.807768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:46.808394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.808669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.808936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.812018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9e66a-b6f7-4fdb-b966-b918dd903779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:46.812632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9e66a-b6f7-4fdb-b966-b918dd903779 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.812905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9e66a-b6f7-4fdb-b966-b918dd903779 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.813169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9e66a-b6f7-4fdb-b966-b918dd903779 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.820292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:46.821734 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:46.824391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9e66a-b6f7-4fdb-b966-b918dd903779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d5c9e66a-b6f7-4fdb-b966-b918dd903779 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:46.829271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:46.830208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:46.831785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:46.835639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 274/542] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:46.836350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9e66a-b6f7-4fdb-b966-b918dd903779 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:46.839057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:46.839964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:46.840860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:46.840860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:46.840860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:46.845906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 269/543] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:46.846753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:46.882739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:46.899398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:46.944887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9e66a-b6f7-4fdb-b966-b918dd903779 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:46.953383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad27764-0a4d-40d9-93eb-6fffabff67b9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:46.979056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88d3b2-e1c9-47cd-8452-e62fb29ba6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:46.979649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88d3b2-e1c9-47cd-8452-e62fb29ba6e0 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.979919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88d3b2-e1c9-47cd-8452-e62fb29ba6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.980183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88d3b2-e1c9-47cd-8452-e62fb29ba6e0 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.980817 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88d3b2-e1c9-47cd-8452-e62fb29ba6e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:46.982185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88d3b2-e1c9-47cd-8452-e62fb29ba6e0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:46.986631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441c24-57fb-4b71-a921-f33b2458070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:46.987265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441c24-57fb-4b71-a921-f33b2458070c None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:46.987535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441c24-57fb-4b71-a921-f33b2458070c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:46.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441c24-57fb-4b71-a921-f33b2458070c None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:46.988392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441c24-57fb-4b71-a921-f33b2458070c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:46.989788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441c24-57fb-4b71-a921-f33b2458070c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:46.997884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 275/544] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:12:46 2022] POST /identity/v3/services => generated 240 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:46.998722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88d3b2-e1c9-47cd-8452-e62fb29ba6e0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.010527 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 270/545] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:12:46 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:47.010527 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441c24-57fb-4b71-a921-f33b2458070c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.158363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:47.164276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88d3b2-e1c9-47cd-8452-e62fb29ba6e0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.185231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:47.198189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441c24-57fb-4b71-a921-f33b2458070c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.215889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.216772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.217195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.217633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.218476 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:47.218921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:47.220567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:47.234043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 276/546] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:47.238356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.240443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.241321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.241667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.242002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.242800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:47.243102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:47.245398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:47.257764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 271/547] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:47.258765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.362389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d482e-fcac-48a2-a561-36118a8b49c2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.376467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85f939-f57f-496f-9ac2-a3e5d2af251f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.401403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f2b2a-cf09-4324-9f1c-6b6962f7f76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.402142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f2b2a-cf09-4324-9f1c-6b6962f7f76f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.402585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f2b2a-cf09-4324-9f1c-6b6962f7f76f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.402914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f2b2a-cf09-4324-9f1c-6b6962f7f76f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.414732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.415031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f2b2a-cf09-4324-9f1c-6b6962f7f76f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:47.415285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.415562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.415833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-983f2b2a-cf09-4324-9f1c-6b6962f7f76f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:47.420350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:47.421284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:47.422131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 277/548] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:47.423457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f2b2a-cf09-4324-9f1c-6b6962f7f76f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.427802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:47.430078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:47.440063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:47.447670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:47.448867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:47.449955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:47.449955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:47.449955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:47.454118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 272/549] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:47.458936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.474607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:47.549150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f2b2a-cf09-4324-9f1c-6b6962f7f76f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.592244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.592875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.593154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.593424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.596036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce031d52-dd95-4164-b7e5-6f5def4a4075 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.603338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:47.605596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:47.621717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:47.622740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 278/550] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:47.628755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.644424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.645811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.646289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.647183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.648322 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:47.648997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:47.650917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:47.663147 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 273/551] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:47.664317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.735309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:47.745545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce1377-807f-49fb-a0c7-5bce3edf3a33 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.782936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e16c-b7cb-41c7-848e-fe89a8349b7c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.786691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:47.791173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.791868 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.792436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.792436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.793201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:47.793569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:47.795079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:47.807966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 279/552] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:47.812652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.832783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.833678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.834579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.835026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.836228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:47.836706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:47.839085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:47.862318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 274/553] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:47.868750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:47.921980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b39f2-6dd4-4b66-9f3e-f3c302d00cd9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:47.961376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:47.962619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:47.963127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:47.964182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:47.981332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:47.982881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:47.999115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:48.000134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:48.001630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:48.001630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:48.001630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:48.001630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 280/554] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:48.001630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.010122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec3ea2-3ab2-4266-8b19-2e7c445c114c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.015408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:48.058217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.059677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.060501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.060965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.066648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:48.075038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.076752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:48.096098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:48.097058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 10:12:48.098343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:48.104560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 275/555] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:48.104560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.136374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961378e-ed2f-4ec5-9fec-8ec90f58f9a2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.182838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af46aee-d846-46ae-8114-92371bb4dc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.183709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af46aee-d846-46ae-8114-92371bb4dc4b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.184122 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af46aee-d846-46ae-8114-92371bb4dc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.184604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af46aee-d846-46ae-8114-92371bb4dc4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.212486 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6a3ff-fcae-410c-8bf3-9258db7b417f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.249313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.249973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.250424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.250791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.251536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:48.251936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.253417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.266603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 276/556] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:48.267574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.276159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af46aee-d846-46ae-8114-92371bb4dc4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.278450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af46aee-d846-46ae-8114-92371bb4dc4b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.292853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:48.314391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:48.378956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb62654-61bf-4286-be1e-a9238e3ef4d8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.406997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 281/557] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:12:48.408275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af46aee-d846-46ae-8114-92371bb4dc4b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.415798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.416405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.416681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.416950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.417735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:48.418044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.419995 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.445700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 277/558] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:48.445700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.526236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af46aee-d846-46ae-8114-92371bb4dc4b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.538820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:48.557823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:48.563045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4e61d-c56b-44e4-ac37-efaa4b94f7a0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.574190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.575326 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.575655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.575927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.586442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.587970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.603298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:48.603757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:48.604772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 282/559] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:48.606286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.607683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.608129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.608561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.621678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.623667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:48.641866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:48.642888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:48.649204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:48.649204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:48.649204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:48.649204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 278/560] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:48.649204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.728591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e33891-e075-4a24-a616-5984481ddfea None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.756294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2de74-732b-457a-b660-013af90f3409 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.764212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.765170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.765659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.766120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.778253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.780067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.793019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.793688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.793896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.794166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.794924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:48.795227 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.796658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:48.798409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:48.799584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:48.800470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:48.800470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:48.800470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:48.802841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 283/561] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:48.804125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.817410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 279/562] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:48.818470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.829326 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:48.831143 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:48.919137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915abf15-aff4-4ff3-b2da-394b7fdbc549 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.936324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508141e1-a9ca-46b4-a41e-ee04eea76787 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:48.956999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.958523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.959160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.959640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.960483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:48.960977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.962920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.973619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:48.974220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:48.974525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:48.974807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:48.975445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:48.975753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:48.977108 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:48.981974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 284/563] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:48.983211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:48.990226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 280/564] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:48.991060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.061844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:49.077562 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:49.100583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c6a6d-aca7-49d7-9cee-f7aa8e56c179 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.110180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b1928-9953-4b9d-b1b2-39d7061aeebb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.139131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.140164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.140671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.141136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.147594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.148247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.148526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.148799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.155509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.157109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.157746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.158654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:49.173265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:49.174270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:49.175253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:49.175253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:49.175253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:49.175253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:49.175253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:49.175253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:49.175253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:49.175253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:49.180006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 281/565] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:49.180998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:49.181972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:49.183012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:49.183012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:49.183012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:49.183186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.185270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 285/566] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:49.186627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.290687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789657d9-1f35-4584-8dad-da9a814ab25e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.296381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c05c9-560e-49fd-824a-b93af1327fe2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.318285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:49.325893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:49.326207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe219108-ca46-470a-9d6a-0622994224e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.326949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219108-ca46-470a-9d6a-0622994224e6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.327254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219108-ca46-470a-9d6a-0622994224e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.327528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219108-ca46-470a-9d6a-0622994224e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.331140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.331734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.332015 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.332287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.333005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:49.333305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.334522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe219108-ca46-470a-9d6a-0622994224e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.334766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.335932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe219108-ca46-470a-9d6a-0622994224e6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.346541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 286/567] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fe219108-ca46-470a-9d6a-0622994224e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:49.347086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:49.348819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:49.349837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:49.349837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:49.349837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:49.349837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:49.349837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:49.349837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:49.349837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:49.349837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:49.350275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.350766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 282/568] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:49.351657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5130c-4542-43d0-b8a7-f9d25e434bc3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.351934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5130c-4542-43d0-b8a7-f9d25e434bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.352206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5130c-4542-43d0-b8a7-f9d25e434bc3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.353584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 283/569] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:49.354615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ada9e-9f10-4cb2-8d82-4d99c995df13 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.354894 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ada9e-9f10-4cb2-8d82-4d99c995df13 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.355164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ada9e-9f10-4cb2-8d82-4d99c995df13 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.356340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 284/570] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:49.357028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ada9e-9f10-4cb2-8d82-4d99c995df13 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.453274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a04fe-6e1a-47c6-a0cd-20929c33bfcc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.458533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ada9e-9f10-4cb2-8d82-4d99c995df13 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.487195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee4d40-91f0-4ef9-83b9-f7aed8556a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.487789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee4d40-91f0-4ef9-83b9-f7aed8556a54 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.488065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee4d40-91f0-4ef9-83b9-f7aed8556a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.488333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee4d40-91f0-4ef9-83b9-f7aed8556a54 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.489610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee4d40-91f0-4ef9-83b9-f7aed8556a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:49.489908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee4d40-91f0-4ef9-83b9-f7aed8556a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.491352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee4d40-91f0-4ef9-83b9-f7aed8556a54 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.492756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.493338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.493608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.493872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.494605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=9d52624718de4457adc44e9965cd7a09) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:49.494907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.496283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.501856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 285/571] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:49 2022] GET /identity/v3/role_assignments?user.id=9d52624718de4457adc44e9965cd7a09&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:49.506965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 287/572] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:49.507155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.507248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497490c8-c658-40a5-8423-97600786f64a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.507248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497490c8-c658-40a5-8423-97600786f64a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.507248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497490c8-c658-40a5-8423-97600786f64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.533388 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-497490c8-c658-40a5-8423-97600786f64a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:49.534635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-497490c8-c658-40a5-8423-97600786f64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:49.573046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:49.588456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:49.616767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 288/573] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:49.618045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830bc1-7595-4cd8-90c1-d9823b8e0b96 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.618345 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830bc1-7595-4cd8-90c1-d9823b8e0b96 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.618617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830bc1-7595-4cd8-90c1-d9823b8e0b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.649276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c5830bc1-7595-4cd8-90c1-d9823b8e0b96 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:49.649529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bc69d-5e5b-400f-a08c-23c89c383c75 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.651144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c5830bc1-7595-4cd8-90c1-d9823b8e0b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:49.690239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.690990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.691265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.691531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.692953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:49.693252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.695353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.710648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 286/574] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:49.711341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.731846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 289/575] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:49.735165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5830bc1-7595-4cd8-90c1-d9823b8e0b96 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.851038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec131caa-90da-418a-9758-e70498d07f73 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.853208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5830bc1-7595-4cd8-90c1-d9823b8e0b96 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:49.868825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:49.870422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:49.894349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e625dd1-1b2b-4fd5-b190-bbb50927209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.895671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e625dd1-1b2b-4fd5-b190-bbb50927209d None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.895971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e625dd1-1b2b-4fd5-b190-bbb50927209d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.896242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e625dd1-1b2b-4fd5-b190-bbb50927209d None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/cf85319814ff4478b40b8a2e28e054ed/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.898216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:49.898374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e625dd1-1b2b-4fd5-b190-bbb50927209d None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=cf85319814ff4478b40b8a2e28e054ed, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.899208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:49.899686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:49.899806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e625dd1-1b2b-4fd5-b190-bbb50927209d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.900268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:49.910143 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:49.913284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:49.917983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 290/576] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:12:49 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/cf85319814ff4478b40b8a2e28e054ed/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:49.918928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e625dd1-1b2b-4fd5-b190-bbb50927209d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:49.927065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:49.928340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:49.929384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:49.929384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:49.929384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:49.929384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:49.929384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:49.929384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:49.929384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:49.929384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:49.932109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 287/577] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:49.942438 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.064561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b80c-ac8e-4d75-be40-d8bedb664ed8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.107107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e625dd1-1b2b-4fd5-b190-bbb50927209d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.115886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.116506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.116785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.117430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.118148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:50.118812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.120578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.142424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 288/578] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:50.143382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.148438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.149177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.149453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.149721 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.150337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.151737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:50.162012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:50.163161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:50.164644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 291/579] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Oct 17 10:12:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:50.165677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.185671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:50.213156 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:50.269775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847ace9-43ea-4c96-9031-cce50f863edc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.288259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2ef2c-c962-47b9-b6f4-bac3f8fb9e44 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.306410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.307035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.307308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.307573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.308154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.309499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:50.320502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:50.321573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:50.323273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:50.330345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 289/580] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:50 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:50.330345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.330978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cade3-2c2a-42c6-b105-46c2cc513934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.331747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cade3-2c2a-42c6-b105-46c2cc513934 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.332120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cade3-2c2a-42c6-b105-46c2cc513934 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.333011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cade3-2c2a-42c6-b105-46c2cc513934 None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/8b862b2975a14e01a5aec3aa936e9375/roles/4045fa40bdd54e9f889b47e1fb2704f5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.349552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cade3-2c2a-42c6-b105-46c2cc513934 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=8b862b2975a14e01a5aec3aa936e9375, role_id=4045fa40bdd54e9f889b47e1fb2704f5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.351192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cade3-2c2a-42c6-b105-46c2cc513934 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.369273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 292/581] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:12:50 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/8b862b2975a14e01a5aec3aa936e9375/roles/4045fa40bdd54e9f889b47e1fb2704f5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:12:50.375328 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cade3-2c2a-42c6-b105-46c2cc513934 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.433186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:50.440133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a40af-10d5-476d-bcfb-8db61b00a8a7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.481439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.482304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.482859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.483282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.484127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:50.484586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.486097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.492294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 290/582] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Oct 17 10:12:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:50.499889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.563618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:50.593589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cade3-2c2a-42c6-b105-46c2cc513934 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.618904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aac8a-d9c6-4c64-ac43-80c769dfc36b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.656904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.657428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.657706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.658055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.658822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:50.659123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.661090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.661762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.662717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.663288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.663769 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.664643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] RBAC: Adding query filter params (name=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:50.665153 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.666521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 291/583] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Oct 17 10:12:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:50.667050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.671856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 293/584] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:12:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:50.672301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.678738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.702158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:50.788562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22291f00-ab0f-4107-9426-2b053c4ecb7b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.819096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d9502-c0e8-4c1a-b7bc-3d6eb561d73d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:50.858133 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.858945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.859284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.859560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] RBAC: Adding query filter params (type=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:50.860494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.861855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.868643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 294/585] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:12:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:50.869035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299482df-0475-4264-8079-f056390a2827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:50.869646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299482df-0475-4264-8079-f056390a2827 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:50.869921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299482df-0475-4264-8079-f056390a2827 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:50.870189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299482df-0475-4264-8079-f056390a2827 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:50.870991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299482df-0475-4264-8079-f056390a2827 None admin] RBAC: Adding query filter params (interface=public, service_id=0e5eeebcac4248ab9aeacf9468700b28, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:50.871294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299482df-0475-4264-8079-f056390a2827 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:50.872633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299482df-0475-4264-8079-f056390a2827 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:50.884006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.891260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 292/586] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Oct 17 10:12:50 2022] GET /identity/v3/endpoints?service_id=0e5eeebcac4248ab9aeacf9468700b28&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:50.897096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299482df-0475-4264-8079-f056390a2827 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:50.922540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:51.039935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452b6f9-4646-4484-9254-48fb59a3f618 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:51.056776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:51.082065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299482df-0475-4264-8079-f056390a2827 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:51.088147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:51.089021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:51.089516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:51.090002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:51.090942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] RBAC: Adding query filter params (interface=public, service_id=49fc133cbfff4727ba39f79e973a6f61, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:51.091471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:51.093557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:51.103695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 295/587] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Oct 17 10:12:50 2022] GET /identity/v3/endpoints?service_id=49fc133cbfff4727ba39f79e973a6f61&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:51.109999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:51.152375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17b34c-10a0-4897-8be4-9cce16acad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:51.152375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17b34c-10a0-4897-8be4-9cce16acad02 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:51.152698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17b34c-10a0-4897-8be4-9cce16acad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:51.152962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17b34c-10a0-4897-8be4-9cce16acad02 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:51.153489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17b34c-10a0-4897-8be4-9cce16acad02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:51.158648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17b34c-10a0-4897-8be4-9cce16acad02 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:51.165095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 293/588] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:12:50 2022] GET /identity/v3/services => generated 772 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:51.211391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:51.243843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4b09-5e2b-4b67-968c-3e772b3999ab None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:51.295446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f4edf-e9c8-43b9-920b-4bb0c262e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:51.296058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4edf-e9c8-43b9-920b-4bb0c262e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:51.296335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4edf-e9c8-43b9-920b-4bb0c262e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:51.296607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4edf-e9c8-43b9-920b-4bb0c262e3f2 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:51.297218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f4edf-e9c8-43b9-920b-4bb0c262e3f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:51.298623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f4edf-e9c8-43b9-920b-4bb0c262e3f2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:51.307377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 296/589] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:12:51 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:52.410180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3f634-ed94-49d8-bc2a-862d90c6e422 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:52.411717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3f634-ed94-49d8-bc2a-862d90c6e422 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:52.412211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3f634-ed94-49d8-bc2a-862d90c6e422 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:52.413786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 294/590] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:52.422877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31bfd4-7b2f-4ed2-98e1-4f07febcee02 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:52.423322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31bfd4-7b2f-4ed2-98e1-4f07febcee02 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:52.423400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31bfd4-7b2f-4ed2-98e1-4f07febcee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:52.446902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4c31bfd4-7b2f-4ed2-98e1-4f07febcee02 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:52.448131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4c31bfd4-7b2f-4ed2-98e1-4f07febcee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:52.493878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64788143-17da-4cb3-a60e-e48563212cc1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:52.494539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64788143-17da-4cb3-a60e-e48563212cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:52.495023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64788143-17da-4cb3-a60e-e48563212cc1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:52.497869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 295/591] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:52.501966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb54f9-3a21-4ef9-9276-28202ac07557 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:52.502591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb54f9-3a21-4ef9-9276-28202ac07557 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:52.503101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb54f9-3a21-4ef9-9276-28202ac07557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:52.534825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eceb54f9-3a21-4ef9-9276-28202ac07557 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:52.537199 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eceb54f9-3a21-4ef9-9276-28202ac07557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:52.549085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 297/592] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:52.643625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 296/593] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:12:52.951096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31bfd4-7b2f-4ed2-98e1-4f07febcee02 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.028632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb54f9-3a21-4ef9-9276-28202ac07557 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.041168 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:53.091327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31bfd4-7b2f-4ed2-98e1-4f07febcee02 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:53.136510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:53.137147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.137424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.137690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.138263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:53.139840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:53.159866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:53.161546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:12:53.162592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:53.163664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:53.165952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 298/594] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Oct 17 10:12:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:53.171319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.197629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb54f9-3a21-4ef9-9276-28202ac07557 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:53.244210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca66646-dde6-4c85-8ecf-a78db58f8a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:53.244754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca66646-dde6-4c85-8ecf-a78db58f8a94 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.245027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca66646-dde6-4c85-8ecf-a78db58f8a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.245296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca66646-dde6-4c85-8ecf-a78db58f8a94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.256570 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca66646-dde6-4c85-8ecf-a78db58f8a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9ca66646-dde6-4c85-8ecf-a78db58f8a94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:53.261700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:53.262682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:53.263564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 297/595] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:53.264809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621055d0-9c66-4dc6-a943-ccc460048ac2 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.264809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621055d0-9c66-4dc6-a943-ccc460048ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.265050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621055d0-9c66-4dc6-a943-ccc460048ac2 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.267131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 298/596] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:53.276498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5707b-1f32-4379-b239-458eab09e4d8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.277004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5707b-1f32-4379-b239-458eab09e4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.277439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5707b-1f32-4379-b239-458eab09e4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.284323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdf101-a809-4030-a8a9-37913ce5f7b6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:53.296590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fce5707b-1f32-4379-b239-458eab09e4d8 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:53.298049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fce5707b-1f32-4379-b239-458eab09e4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:53.320901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:53.321507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.321778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.322042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.322764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:53.323062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:53.324551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:53.326523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:53.330367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 299/597] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:53.331268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.382693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 299/598] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:53.383926 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5707b-1f32-4379-b239-458eab09e4d8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.438884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8078952-5099-4d88-8ec1-c7d501cd7009 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:53.483159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:53.483854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.484254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.484819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.485551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:53.489942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:53.489942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:53.498796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 300/599] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:53.511495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.531651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:53.558889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5707b-1f32-4379-b239-458eab09e4d8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:53.608932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:53.610036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.610540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.611023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.611963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:53.612589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:53.621606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:53.627795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 300/600] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:53.629299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:53.702149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c16da-e86d-4556-ba77-2273fc4134ac None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:53.749579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:53.750137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.750434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.750702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.767543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:53.769084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:53.788693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:53.798992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:53.800065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:53.801001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:53.801001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:53.801001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:53.801001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 301/601] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:53.807186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.924477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764982b-5011-423d-a981-4f7f1b2ca2c9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:53.928942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fab59-a846-4e6f-b0eb-5a8f39e9d902 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:53.940777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:53.963643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:53.964333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.964612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.965109 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.965862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:53.966352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:53.967872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:53.968009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9c095-c604-4e19-b0fc-eb1f41140128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:53.968674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c095-c604-4e19-b0fc-eb1f41140128 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.969018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c095-c604-4e19-b0fc-eb1f41140128 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.969360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c095-c604-4e19-b0fc-eb1f41140128 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.972152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:53.982263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9c095-c604-4e19-b0fc-eb1f41140128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-95f9c095-c604-4e19-b0fc-eb1f41140128 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:12:53.988712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:53.990351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:53.991411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 301/603] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:53.992324 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 302/603] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:53.993294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720691a2-c991-4c29-a046-b277d2535ec0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:53.993294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720691a2-c991-4c29-a046-b277d2535ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:53.993496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720691a2-c991-4c29-a046-b277d2535ec0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:53.994693 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:53.995525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 302/604] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:54.000498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720691a2-c991-4c29-a046-b277d2535ec0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:54.100732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de832f0-4857-4697-b5cb-31ba20a648d5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:54.127511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720691a2-c991-4c29-a046-b277d2535ec0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:54.135597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d21b50-898a-4b24-99e2-503920d94125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:54.136200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d21b50-898a-4b24-99e2-503920d94125 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.136479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d21b50-898a-4b24-99e2-503920d94125 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.136744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d21b50-898a-4b24-99e2-503920d94125 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.137406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d21b50-898a-4b24-99e2-503920d94125 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:54.137706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d21b50-898a-4b24-99e2-503920d94125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:54.139105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d21b50-898a-4b24-99e2-503920d94125 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:54.149019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 303/605] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:54.150207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e9851-622d-4549-bdd0-45a2e0da0e42 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.150530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e9851-622d-4549-bdd0-45a2e0da0e42 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.150804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e9851-622d-4549-bdd0-45a2e0da0e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.169858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-577e9851-622d-4549-bdd0-45a2e0da0e42 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:54.170951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed19064-fcbe-41a8-b10e-1acfca525190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:54.171097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-577e9851-622d-4549-bdd0-45a2e0da0e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:54.171577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed19064-fcbe-41a8-b10e-1acfca525190 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.171908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed19064-fcbe-41a8-b10e-1acfca525190 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.172180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed19064-fcbe-41a8-b10e-1acfca525190 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.180526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed19064-fcbe-41a8-b10e-1acfca525190 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:54.181958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed19064-fcbe-41a8-b10e-1acfca525190 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6ed19064-fcbe-41a8-b10e-1acfca525190 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:54.194430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:54.195451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:54.196404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 303/606] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:12:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:54.201422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359421e-171d-457d-8fa0-e9e83b5d2291 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.201711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359421e-171d-457d-8fa0-e9e83b5d2291 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.201982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359421e-171d-457d-8fa0-e9e83b5d2291 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.203268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 304/607] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:54.203959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359421e-171d-457d-8fa0-e9e83b5d2291 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:54.206899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:54.248786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 304/608] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:54.249658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e9851-622d-4549-bdd0-45a2e0da0e42 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:54.278734 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:54.338385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359421e-171d-457d-8fa0-e9e83b5d2291 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:54.354581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e9851-622d-4549-bdd0-45a2e0da0e42 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:54.383303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:54.383943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.384219 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.384487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.392221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf61744-eb2d-4043-926f-1b1cadec079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:54.392829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf61744-eb2d-4043-926f-1b1cadec079e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.393108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf61744-eb2d-4043-926f-1b1cadec079e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.393379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf61744-eb2d-4043-926f-1b1cadec079e None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.394103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf61744-eb2d-4043-926f-1b1cadec079e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:54.394437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf61744-eb2d-4043-926f-1b1cadec079e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:54.396260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf61744-eb2d-4043-926f-1b1cadec079e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:54.401504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:54.403559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:54.419820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 305/609] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Oct 17 10:12:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:54.421144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289fa71-e9a8-40e9-9408-e94a8c060898 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.421428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289fa71-e9a8-40e9-9408-e94a8c060898 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.421701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289fa71-e9a8-40e9-9408-e94a8c060898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:54.422314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:54.423274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:54.424204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:54.424204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:54.424204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:54.426625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 305/610] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:54.431382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:54.446216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c289fa71-e9a8-40e9-9408-e94a8c060898 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:54.447446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c289fa71-e9a8-40e9-9408-e94a8c060898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:54.466445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:54.532474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 306/611] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:54.533148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289fa71-e9a8-40e9-9408-e94a8c060898 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:54.559921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f1fe-d425-4275-830e-d1dbb871e8c8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:54.569734 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:54.602213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7c442-2699-4da7-97c6-41fb4b1373da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:54.603050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7c442-2699-4da7-97c6-41fb4b1373da None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.603329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7c442-2699-4da7-97c6-41fb4b1373da None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.603600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7c442-2699-4da7-97c6-41fb4b1373da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.678497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289fa71-e9a8-40e9-9408-e94a8c060898 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:54.687786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e7c442-2699-4da7-97c6-41fb4b1373da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:54.690313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e7c442-2699-4da7-97c6-41fb4b1373da None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:54.724912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:54.726287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.727052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.727526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.728364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:54.729074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:54.731527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:54.751001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 307/612] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:54.752422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:54.773465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:54.803466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 306/613] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:12:54.804351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7c442-2699-4da7-97c6-41fb4b1373da None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:54.815437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:54.868065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512b19-519e-4fd5-a75a-4a5052cb3bab None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:54.904731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:54.905449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.905718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.905986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.906596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:54.907943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:54.915021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7c442-2699-4da7-97c6-41fb4b1373da None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:54.917953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:54.922462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:54.923381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 308/614] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Oct 17 10:12:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:54.924558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:54.959652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:54.960370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:54.960714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:54.961054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:54.969658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:54.971302 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:12:54.984739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:54.985774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 307/615] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Oct 17 10:12:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:54.987281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.012908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.037279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bd985-cdc5-4a52-8d35-e2339f0ac8de None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.057795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.080430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.080579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.080803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.081074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] PATH_INFO: `/v3/services/network` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.081658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.083095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:55.092058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:55.093268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:55.094313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 309/616] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:54 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:55.095452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.108191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc614f22-b20a-4e0f-84e1-e52738b2be28 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.151068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.151686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.151965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.152236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.163710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.165150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:55.179801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:55.180854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:55.181763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:55.181763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:55.181763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:55.181763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 308/617] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:55.181763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.204836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd75f16-bee8-40f3-baac-3210b93b1f63 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.240623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.241467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.241891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.242162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.242814 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:55.243112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.244479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.248961 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 310/618] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Oct 17 10:12:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:55.249744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.262692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.295439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e57c1-00f5-4f0d-bed6-d61f1a54bfa3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.320009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.338921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.339533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.339811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.340251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.341355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:55.341666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.343566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.359036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465e9f0-affe-46a4-a633-0281c8307d2d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.362653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 309/619] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Oct 17 10:12:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:55.363514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.399835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.400767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.401229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.401698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.402350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] RBAC: Adding query filter params (name=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:55.402650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.404008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.414089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 311/620] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:12:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:55.414089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.478265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f01b73-4292-41b4-9469-c9e0b79c359b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.485845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.518002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.518633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.518759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d663-4de7-487e-a2af-6612f47cb427 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.518938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.519188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.520365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:55.520669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.522645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.538969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 310/621] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:55.538969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.558199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.558909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.559246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.559583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.560203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:55.560501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.561845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.567403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 312/622] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Oct 17 10:12:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:55.568197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.590247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.655662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224a317-8938-419e-9637-8b0e7da21262 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.677239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2775cb1-f0da-428f-93f0-0574054a3fe4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.692380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86aaa5-7336-43e3-be6d-09d785654f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.693114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86aaa5-7336-43e3-be6d-09d785654f68 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.693467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86aaa5-7336-43e3-be6d-09d785654f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.693811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86aaa5-7336-43e3-be6d-09d785654f68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.714417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.715026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.715297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.715564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.716188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] RBAC: Adding query filter params (type=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:55.716488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.718439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.720139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.731150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 313/623] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:12:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:55.732878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.767100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86aaa5-7336-43e3-be6d-09d785654f68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.768580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86aaa5-7336-43e3-be6d-09d785654f68 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.813882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.840455 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2879b-7700-4452-9859-215425f08b67 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:55.874424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:55.875047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:55.875322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:55.875588 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:55.885352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 311/624] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:12:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:12:55.885352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86aaa5-7336-43e3-be6d-09d785654f68 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.886879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:55.888415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:55.900278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:55.901384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:55.902404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 314/625] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:55.903006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:55.961249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:55.987008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86aaa5-7336-43e3-be6d-09d785654f68 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.002789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c2d53-77c9-4638-9cd3-3e0732e692d0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.021394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddfe55-f9c6-45b2-8b7c-d3d26c7808e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.021986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddfe55-f9c6-45b2-8b7c-d3d26c7808e8 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.022262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddfe55-f9c6-45b2-8b7c-d3d26c7808e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.022566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddfe55-f9c6-45b2-8b7c-d3d26c7808e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.023221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddfe55-f9c6-45b2-8b7c-d3d26c7808e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.024569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddfe55-f9c6-45b2-8b7c-d3d26c7808e8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.032903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:56.036456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c7fa9-259c-43ad-a12a-936783cfbad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.037222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c7fa9-259c-43ad-a12a-936783cfbad2 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.037584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c7fa9-259c-43ad-a12a-936783cfbad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.037852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c7fa9-259c-43ad-a12a-936783cfbad2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.038853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c7fa9-259c-43ad-a12a-936783cfbad2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.040612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c7fa9-259c-43ad-a12a-936783cfbad2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.045781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 312/626] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:12:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:56.046721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddfe55-f9c6-45b2-8b7c-d3d26c7808e8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:56.078362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 315/627] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:12:55 2022] POST /identity/v3/endpoints => generated 337 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:56.078362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e168b-872a-42f5-96e9-52660dbdffaf None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.078362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e168b-872a-42f5-96e9-52660dbdffaf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.078362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e168b-872a-42f5-96e9-52660dbdffaf None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.078362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 316/628] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:56.078362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e168b-872a-42f5-96e9-52660dbdffaf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:56.221542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddfe55-f9c6-45b2-8b7c-d3d26c7808e8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.231718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e168b-872a-42f5-96e9-52660dbdffaf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.246118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:56.275699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.276846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.276846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.277018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.281958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7d096-2e5a-4e6c-8e66-b7c1d2944ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.282599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7d096-2e5a-4e6c-8e66-b7c1d2944ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.282880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7d096-2e5a-4e6c-8e66-b7c1d2944ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.283154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7d096-2e5a-4e6c-8e66-b7c1d2944ce1 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.283857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7d096-2e5a-4e6c-8e66-b7c1d2944ce1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:56.284166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7d096-2e5a-4e6c-8e66-b7c1d2944ce1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.286182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7d096-2e5a-4e6c-8e66-b7c1d2944ce1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.289025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.290606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.300892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 317/629] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:56.302051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46a50-c3cc-4134-a1d1-6cec9fc73020 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.302349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46a50-c3cc-4134-a1d1-6cec9fc73020 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.302679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46a50-c3cc-4134-a1d1-6cec9fc73020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:12:56.306715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:56.307838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:12:56.309268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:12:56.309268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:12:56.309268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:56.309268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 313/630] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:56.309268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:56.326476 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-28a46a50-c3cc-4134-a1d1-6cec9fc73020 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:56.327684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-28a46a50-c3cc-4134-a1d1-6cec9fc73020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:56.365471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:56.414352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 318/631] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:56.415206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a46a50-c3cc-4134-a1d1-6cec9fc73020 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:56.435759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5d316-d6b7-4b9b-8333-8865acc188ca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.474162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.475087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.475514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.475930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.476895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=cf85319814ff4478b40b8a2e28e054ed) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:56.477466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.479065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.485939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 314/632] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:56 2022] GET /identity/v3/role_assignments?user.id=cf85319814ff4478b40b8a2e28e054ed&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:56.521467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:56.525564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a46a50-c3cc-4134-a1d1-6cec9fc73020 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.565704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.566414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.566800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.567076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.567741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:56.568099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.569574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.583514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 319/633] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:56.589174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:56.702745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca1f7b-bdd3-47ce-a43a-f614df313aec None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.705245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:56.713479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:56.743635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.744275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.744555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.744819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.752032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.753473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:12:56.766573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:56.767707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:56.768790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 315/634] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:12:56.774230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:56.824807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca28246-657e-4451-9a5b-11648b9f9a9c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.862287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed6b2d-bba0-4f0a-b611-74bc5b2cf6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.862936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed6b2d-bba0-4f0a-b611-74bc5b2cf6ac None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.863214 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed6b2d-bba0-4f0a-b611-74bc5b2cf6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.863521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed6b2d-bba0-4f0a-b611-74bc5b2cf6ac None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.864711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed6b2d-bba0-4f0a-b611-74bc5b2cf6ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.866192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed6b2d-bba0-4f0a-b611-74bc5b2cf6ac None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.884334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 320/635] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:12:56 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:12:56.894224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aedc58-4438-487f-8e7a-ce8827a0dd70 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:56.944050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8ca6e-81a2-4363-bf6e-6559418773dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:56.944975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8ca6e-81a2-4363-bf6e-6559418773dc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:56.945450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8ca6e-81a2-4363-bf6e-6559418773dc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:56.945866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8ca6e-81a2-4363-bf6e-6559418773dc None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:56.946769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8ca6e-81a2-4363-bf6e-6559418773dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:56.947302 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8ca6e-81a2-4363-bf6e-6559418773dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:56.949018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8ca6e-81a2-4363-bf6e-6559418773dc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:56.964276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 316/636] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:56.970068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed6b2d-bba0-4f0a-b611-74bc5b2cf6ac None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:56.997957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:57.097108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed6b2d-bba0-4f0a-b611-74bc5b2cf6ac None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:57.137454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f221bf2-5b53-4233-bd47-998062514f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:57.138065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f221bf2-5b53-4233-bd47-998062514f9c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:57.138361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f221bf2-5b53-4233-bd47-998062514f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:57.138639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f221bf2-5b53-4233-bd47-998062514f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:57.139361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f221bf2-5b53-4233-bd47-998062514f9c None admin] RBAC: Adding query filter params (role.id=4045fa40bdd54e9f889b47e1fb2704f5, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=8b862b2975a14e01a5aec3aa936e9375) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:57.139673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f221bf2-5b53-4233-bd47-998062514f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:57.141099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f221bf2-5b53-4233-bd47-998062514f9c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:57.147957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 321/637] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:12:56 2022] GET /identity/v3/role_assignments?user.id=8b862b2975a14e01a5aec3aa936e9375&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=4045fa40bdd54e9f889b47e1fb2704f5 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:58.377664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a52140-4177-4617-9009-6cd68d763fa3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:58.378179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a52140-4177-4617-9009-6cd68d763fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:58.378179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a52140-4177-4617-9009-6cd68d763fa3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:58.380001 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 317/638] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:58.385420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908c08c-9a14-4ec2-a4f8-1b534a98219a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:58.385834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908c08c-9a14-4ec2-a4f8-1b534a98219a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:58.386024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908c08c-9a14-4ec2-a4f8-1b534a98219a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:58.404567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c908c08c-9a14-4ec2-a4f8-1b534a98219a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:58.405738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c908c08c-9a14-4ec2-a4f8-1b534a98219a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:58.480910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 322/639] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:58 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:58.727751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352bc8f-2178-4b84-b2b8-e60212f818ca None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:58.728101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352bc8f-2178-4b84-b2b8-e60212f818ca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:58.728291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352bc8f-2178-4b84-b2b8-e60212f818ca None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:58.729783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 318/640] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:58.735068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fa4a-bf7c-476c-814c-7fde42d87aa4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:58.735612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fa4a-bf7c-476c-814c-7fde42d87aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:58.736085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fa4a-bf7c-476c-814c-7fde42d87aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:58.751863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352bc8f-2178-4b84-b2b8-e60212f818ca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:58.757970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2d37fa4a-bf7c-476c-814c-7fde42d87aa4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:58.759196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2d37fa4a-bf7c-476c-814c-7fde42d87aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:58.772758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:58.779712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:58.857468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 323/641] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:58 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:12:58.876711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352bc8f-2178-4b84-b2b8-e60212f818ca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:58.914193 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:58.915218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:58.915697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:58.916166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:58.916969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:58.918629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:58.928014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:58.929027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:12:58.929906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:12:58.932339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 319/642] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:12:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:58.938418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37fa4a-bf7c-476c-814c-7fde42d87aa4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:59.046814 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37fa4a-bf7c-476c-814c-7fde42d87aa4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:59.098196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:59.099130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.099554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.099994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.100759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:59.101224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:59.102810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:59.107997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 324/643] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:12:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:59.113757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:59.146979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:59.158526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:59.208821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:59.226735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318dc2-3375-4b94-ab17-94c1bd88228d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:59.255129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974589fa-fc15-4822-901d-8c741b8be679 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:59.264665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398a04a-233d-4d3e-bdb5-69d9bb5811ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:59.265287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a04a-233d-4d3e-bdb5-69d9bb5811ae None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.265564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a04a-233d-4d3e-bdb5-69d9bb5811ae None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.265834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a04a-233d-4d3e-bdb5-69d9bb5811ae None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.266513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398a04a-233d-4d3e-bdb5-69d9bb5811ae None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:59.266832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398a04a-233d-4d3e-bdb5-69d9bb5811ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:59.268247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398a04a-233d-4d3e-bdb5-69d9bb5811ae None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:59.279539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 320/644] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:12:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:59.279539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0ae7-1440-4112-a5de-f164dc6d46a9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.279539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0ae7-1440-4112-a5de-f164dc6d46a9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.279539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0ae7-1440-4112-a5de-f164dc6d46a9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.279539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 321/645] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:59.284255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772d1ad-4c22-4162-88b8-f113c02c67f6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.284580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772d1ad-4c22-4162-88b8-f113c02c67f6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.284803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772d1ad-4c22-4162-88b8-f113c02c67f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.308881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3772d1ad-4c22-4162-88b8-f113c02c67f6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:59.310708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3772d1ad-4c22-4162-88b8-f113c02c67f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:59.315549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:59.316622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.317284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.317758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.318595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:59.320971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:59.334697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:59.335829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:12:59.336683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:59.340644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 325/646] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:12:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:59.346668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:59.398775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 322/647] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:59 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:12:59.400261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0138b-fe96-453d-8a23-38e72f69549c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.400552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0138b-fe96-453d-8a23-38e72f69549c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.400828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0138b-fe96-453d-8a23-38e72f69549c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.402307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 323/648] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:12:59.407693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf8c6f-0df0-48c6-91f5-7d24895c2963 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.407972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf8c6f-0df0-48c6-91f5-7d24895c2963 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.408245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf8c6f-0df0-48c6-91f5-7d24895c2963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.432289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-81bf8c6f-0df0-48c6-91f5-7d24895c2963 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:12:59.433525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-81bf8c6f-0df0-48c6-91f5-7d24895c2963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:12:59.441998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:59.483350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065010f-f2e3-49fb-9e84-1ce41f36a2dd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:59.517839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 324/649] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:12:59 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:12:59.532970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:59.533953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.534492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.535010 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.535868 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:59.536425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:59.538029 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:59.543444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 326/650] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:12:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:12:59.549229 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf8c6f-0df0-48c6-91f5-7d24895c2963 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:59.567458 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:59.655128 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf8c6f-0df0-48c6-91f5-7d24895c2963 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:59.678420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:59.690603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:59.691261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.692090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.692398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.693060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:12:59.693365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:59.694758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:59.700477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 325/651] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:12:59 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:12:59.795143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71118575-24a9-4742-987f-385fb3b51bd8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:59.831376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:59.837722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:12:59.838395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:12:59.838686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:12:59.838962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:12:59.840514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:12:59.841931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:12:59.845679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:12:59.855551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:12:59.856622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:12:59.857551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:12:59.866729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 327/652] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Oct 17 10:12:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:12:59.868482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:12:59.934839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:12:59.963575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513a2ec-50f0-4a00-9c52-40ddbd18e040 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:12:59.988006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e35fe-a58c-4f8d-a5c7-f1355d461fca None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.003811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86a0e4-07e3-4557-9321-586fdbd76b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.004563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a0e4-07e3-4557-9321-586fdbd76b5a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.004805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a0e4-07e3-4557-9321-586fdbd76b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.005094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a0e4-07e3-4557-9321-586fdbd76b5a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.018923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86a0e4-07e3-4557-9321-586fdbd76b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ff86a0e4-07e3-4557-9321-586fdbd76b5a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:13:00.024887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:00.025827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:00.026737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 326/653] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Oct 17 10:12:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:00.031799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86a0e4-07e3-4557-9321-586fdbd76b5a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:00.035052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.036631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.037165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.037633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.038500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:00.039036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.040686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.046610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 328/654] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:12:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:00.052793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:00.141668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86a0e4-07e3-4557-9321-586fdbd76b5a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.153257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:00.184918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.185666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.186011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.186372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.187633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:00.188327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.190523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.192726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:00.199877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06f70d-1a5e-4c8d-b315-ecde6aaa7e45 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.204758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 327/655] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:00.213014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:00.242370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.243182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.243468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.243740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.244730 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:00.245034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.250370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.253818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 329/656] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:00.259604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:00.334645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f984a7c-ebfa-4cf9-818d-eca2eb638673 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.371282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.371760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.372038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.372306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.384528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.386058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.398690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e7089-4130-45bd-88a1-8c0832b3d948 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:00.401752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:00.403078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:00.403992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:00.403992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:00.403992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:00.407509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 328/657] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:00.408397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:00.433444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:00.442942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.443592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.443805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.444138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.444771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] RBAC: Adding query filter params (interface=public, service_id=97487f758c764ba8b50b73087f4c2df3, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:00.445078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.446510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.452328 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 330/658] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/endpoints?service_id=97487f758c764ba8b50b73087f4c2df3&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:00.457785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:00.479178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:00.537251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd0a4b-84eb-4663-9c13-b692e28d7cdb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.578317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.578949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.579224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.579548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.580452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:00.580756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.582898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.584862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b28f2d-56ea-40bb-b646-8120a18c3074 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.596002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 329/659] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:00.603155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:00.623013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057029-3b04-425b-aa5e-0dcd6430ac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.623943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057029-3b04-425b-aa5e-0dcd6430ac99 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.624372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057029-3b04-425b-aa5e-0dcd6430ac99 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.624786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057029-3b04-425b-aa5e-0dcd6430ac99 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.625584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe057029-3b04-425b-aa5e-0dcd6430ac99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.627126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe057029-3b04-425b-aa5e-0dcd6430ac99 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.634040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 331/660] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/services => generated 1029 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:13:00.733286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3cbc1-b613-43dc-b53c-0e2b8b0edafb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.745195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:00.774974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.775860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.776357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.776778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.785540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.787424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:13:00.800796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:00.802608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 10:13:00.803836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:00.806121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 330/661] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:00.811803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057029-3b04-425b-aa5e-0dcd6430ac99 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:00.873575 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:00.921055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057029-3b04-425b-aa5e-0dcd6430ac99 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:00.956232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:00.956843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:00.957205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:00.957500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:00.958208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:00.958541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:00.959980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:00.979016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 332/662] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:00.987521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:01.101759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6e7ca-3b87-4cbe-9196-e12e1a56b3c4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:01.138438 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:01.139070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:01.139349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:01.139636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:01.152243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:01.153696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:01.168852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:01.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:01.170788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:01.170788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:01.170788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:01.170788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 331/663] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:01.177326 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:01.283732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1fa1-ecd4-4183-b7b6-57aa15048db7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:01.299631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:01.322106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:01.323060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:01.323566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:01.324033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:01.324860 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:01.325360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:01.326964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:01.339618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 333/664] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:01.345358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:01.370026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:01.460814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5be9ca-1dd6-4c6c-ad0f-244a1612f547 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:01.497045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782d075-340b-4256-bd14-955e52317919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:01.497718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782d075-340b-4256-bd14-955e52317919 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:01.498009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782d075-340b-4256-bd14-955e52317919 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:01.498281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782d075-340b-4256-bd14-955e52317919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:01.505769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782d075-340b-4256-bd14-955e52317919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:01.507339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782d075-340b-4256-bd14-955e52317919 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b782d075-340b-4256-bd14-955e52317919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:13:01.519119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:01.520654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:01.522709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 332/665] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:13:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:01.526755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:01.529812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b159-e618-42c9-a518-796992f70e42 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:01.530102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b159-e618-42c9-a518-796992f70e42 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:01.530417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b159-e618-42c9-a518-796992f70e42 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:01.533126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 333/666] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:01.537603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e15c-901d-44d5-95bb-39b79de350e9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:01.537813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e15c-901d-44d5-95bb-39b79de350e9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:01.538077 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e15c-901d-44d5-95bb-39b79de350e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:01.558697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8558e15c-901d-44d5-95bb-39b79de350e9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:01.560775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8558e15c-901d-44d5-95bb-39b79de350e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:01.644545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 334/667] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:01.660620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac940092-485b-40ef-9f75-1d7251fe2ad5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:01.701965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:01.702993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:01.703492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:01.704003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:01.704884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:01.705384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:01.707074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:01.721438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 334/668] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:01.727424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e15c-901d-44d5-95bb-39b79de350e9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:01.740575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:01.833363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e15c-901d-44d5-95bb-39b79de350e9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:01.869986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6cde1-63b5-4ff9-add8-14a8fb74846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:01.870897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6cde1-63b5-4ff9-add8-14a8fb74846c None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:01.871349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6cde1-63b5-4ff9-add8-14a8fb74846c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:01.871768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6cde1-63b5-4ff9-add8-14a8fb74846c None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/8b862b2975a14e01a5aec3aa936e9375/roles/1c2e3fb06553458f8c24ed50ac46ed5e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:01.874063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6cde1-63b5-4ff9-add8-14a8fb74846c None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=8b862b2975a14e01a5aec3aa936e9375, role_id=1c2e3fb06553458f8c24ed50ac46ed5e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:01.875717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6cde1-63b5-4ff9-add8-14a8fb74846c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:01.889696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 335/669] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:13:01 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/8b862b2975a14e01a5aec3aa936e9375/roles/1c2e3fb06553458f8c24ed50ac46ed5e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:13:01.943040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:01.985098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:02.027774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bdaa2-1379-41d6-b1bc-54ff45ee09cf None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:02.028004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bdaa2-1379-41d6-b1bc-54ff45ee09cf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:02.028267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bdaa2-1379-41d6-b1bc-54ff45ee09cf None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:02.029890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 336/670] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:02.035483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1afd6-9f33-4498-b655-41857251fcb3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:02.035911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1afd6-9f33-4498-b655-41857251fcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:02.036117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1afd6-9f33-4498-b655-41857251fcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:02.056751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-22e1afd6-9f33-4498-b655-41857251fcb3 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:02.058007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22e1afd6-9f33-4498-b655-41857251fcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:02.067733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058434a4-c03f-40ef-bd5a-a7a478945d3e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:02.118819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd6d9e-b396-4e7f-8e8a-8f7e1a79b3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:02.119352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6d9e-b396-4e7f-8e8a-8f7e1a79b3b5 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:02.119633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6d9e-b396-4e7f-8e8a-8f7e1a79b3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:02.119906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6d9e-b396-4e7f-8e8a-8f7e1a79b3b5 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:02.120850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd6d9e-b396-4e7f-8e8a-8f7e1a79b3b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:02.122561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd6d9e-b396-4e7f-8e8a-8f7e1a79b3b5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:02.139221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:02.145017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 337/671] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:02 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:02.149372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 335/672] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:13:01 2022] POST /identity/v3/services => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:02.415384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1afd6-9f33-4498-b655-41857251fcb3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:02.522138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1afd6-9f33-4498-b655-41857251fcb3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:02.563751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74179e2-a3f1-497b-9b1e-25a5265647d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:02.564416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74179e2-a3f1-497b-9b1e-25a5265647d0 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:02.564697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74179e2-a3f1-497b-9b1e-25a5265647d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:02.564965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74179e2-a3f1-497b-9b1e-25a5265647d0 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:02.565620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74179e2-a3f1-497b-9b1e-25a5265647d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:02.567114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74179e2-a3f1-497b-9b1e-25a5265647d0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:02.583750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 338/673] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:13:02 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:03.081294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa7d2-f8cd-48b7-8054-507a2c44ba75 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:03.081754 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa7d2-f8cd-48b7-8054-507a2c44ba75 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:03.082031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa7d2-f8cd-48b7-8054-507a2c44ba75 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:03.084162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 336/674] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:03.089678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bd2d4-6eb9-4ffc-abbf-762696918576 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:03.090077 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bd2d4-6eb9-4ffc-abbf-762696918576 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:03.090485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bd2d4-6eb9-4ffc-abbf-762696918576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:03.109931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c91bd2d4-6eb9-4ffc-abbf-762696918576 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:03.111136 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c91bd2d4-6eb9-4ffc-abbf-762696918576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:03.165529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:03.193427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 339/675] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:03.466627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffa7d2-f8cd-48b7-8054-507a2c44ba75 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:03.570997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:03.578909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffa7d2-f8cd-48b7-8054-507a2c44ba75 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:03.615137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:03.615763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:03.616039 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:03.616311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:03.616892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:03.618258 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:03.627243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:03.628186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:03.629132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 337/676] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Oct 17 10:13:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:03.634721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bd2d4-6eb9-4ffc-abbf-762696918576 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:03.741164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bd2d4-6eb9-4ffc-abbf-762696918576 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:03.776536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:03.777134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:03.777406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:03.777673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:03.778294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:03.778619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:03.779961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:03.785984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 340/677] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:13:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:03.791414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:03.897761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429229-de1d-4d6d-9fd1-e2d4d009f6bc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:03.933125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32b8b6-079b-406b-ac74-ecabaf8ea3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:03.933735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32b8b6-079b-406b-ac74-ecabaf8ea3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:03.934015 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32b8b6-079b-406b-ac74-ecabaf8ea3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:03.934290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32b8b6-079b-406b-ac74-ecabaf8ea3d7 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:03.934955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32b8b6-079b-406b-ac74-ecabaf8ea3d7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:03.935257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32b8b6-079b-406b-ac74-ecabaf8ea3d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:03.936685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32b8b6-079b-406b-ac74-ecabaf8ea3d7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:03.942715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 338/678] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:13:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:03.948564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:04.000828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:04.063870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58b8c1-7fd3-442b-84cf-88dc4ae6f798 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:04.100288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f142d37-f518-419e-b207-01e89a1e1d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:04.100884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142d37-f518-419e-b207-01e89a1e1d6f None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.101158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142d37-f518-419e-b207-01e89a1e1d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.101423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142d37-f518-419e-b207-01e89a1e1d6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.102091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142d37-f518-419e-b207-01e89a1e1d6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:04.103473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142d37-f518-419e-b207-01e89a1e1d6f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:04.112921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9edee-5f8c-4343-897e-9937423b4f79 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.113942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9edee-5f8c-4343-897e-9937423b4f79 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.114444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9edee-5f8c-4343-897e-9937423b4f79 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.115890 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 339/679] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:04.121230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eb696-64dc-4402-b7f9-50901bf70e77 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.121515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eb696-64dc-4402-b7f9-50901bf70e77 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.121794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eb696-64dc-4402-b7f9-50901bf70e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.125764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 341/680] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:13:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:04.141549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f41eb696-64dc-4402-b7f9-50901bf70e77 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:04.142886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f41eb696-64dc-4402-b7f9-50901bf70e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:04.163178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:04.226904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 340/681] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:04.421178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c1e7a-c809-439d-84b3-9f52bbe3fb83 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.421362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c1e7a-c809-439d-84b3-9f52bbe3fb83 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.421628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c1e7a-c809-439d-84b3-9f52bbe3fb83 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.423256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 342/682] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:04.428547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10feae0-c924-4be7-acb8-85c2a85e8dbb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.428830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10feae0-c924-4be7-acb8-85c2a85e8dbb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.429099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10feae0-c924-4be7-acb8-85c2a85e8dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.448060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e10feae0-c924-4be7-acb8-85c2a85e8dbb None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:04.449227 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e10feae0-c924-4be7-acb8-85c2a85e8dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:04.499420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c1e7a-c809-439d-84b3-9f52bbe3fb83 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:04.531698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 341/683] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:04.617341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c1e7a-c809-439d-84b3-9f52bbe3fb83 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:04.618924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:04.654186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ae610-0841-4ea9-bb18-055ba23888ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:04.654848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae610-0841-4ea9-bb18-055ba23888ae None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.655101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae610-0841-4ea9-bb18-055ba23888ae None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.655371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae610-0841-4ea9-bb18-055ba23888ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.666303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ae610-0841-4ea9-bb18-055ba23888ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-873ae610-0841-4ea9-bb18-055ba23888ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:13:04.672296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:04.673283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 10:13:04.674188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:04.677341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 343/684] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Oct 17 10:13:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:04.683353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10feae0-c924-4be7-acb8-85c2a85e8dbb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:04.796375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10feae0-c924-4be7-acb8-85c2a85e8dbb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:04.797908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:04.811560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ae610-0841-4ea9-bb18-055ba23888ae None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:04.837916 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fef60-99bb-41a2-ab0d-c5797cfe5ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:04.838878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fef60-99bb-41a2-ab0d-c5797cfe5ecd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.839176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fef60-99bb-41a2-ab0d-c5797cfe5ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.839773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fef60-99bb-41a2-ab0d-c5797cfe5ecd None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.840454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fef60-99bb-41a2-ab0d-c5797cfe5ecd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:04.840928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fef60-99bb-41a2-ab0d-c5797cfe5ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:04.843021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fef60-99bb-41a2-ab0d-c5797cfe5ecd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:04.854412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 342/685] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Oct 17 10:13:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:04.856025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac2ca0-28a3-40ad-a854-bc45429ca285 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.856322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac2ca0-28a3-40ad-a854-bc45429ca285 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.856599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac2ca0-28a3-40ad-a854-bc45429ca285 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.858116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 343/686] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:04.860845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac2ca0-28a3-40ad-a854-bc45429ca285 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:04.924048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ae610-0841-4ea9-bb18-055ba23888ae None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:04.960156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4efa1-1ee9-4bf4-acdf-07dc1c02692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:04.960784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4efa1-1ee9-4bf4-acdf-07dc1c02692c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.961060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4efa1-1ee9-4bf4-acdf-07dc1c02692c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.961332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4efa1-1ee9-4bf4-acdf-07dc1c02692c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.961922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4efa1-1ee9-4bf4-acdf-07dc1c02692c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:04.963304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4efa1-1ee9-4bf4-acdf-07dc1c02692c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:04.968505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac2ca0-28a3-40ad-a854-bc45429ca285 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4cc4efa1-1ee9-4bf4-acdf-07dc1c02692c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:04.972318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:04.974065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:04.975364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 344/687] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:13:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:04.976506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7be1b-b932-4809-a012-18957b5b5d3b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:04.976506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7be1b-b932-4809-a012-18957b5b5d3b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:04.976506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7be1b-b932-4809-a012-18957b5b5d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:04.994905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-aba7be1b-b932-4809-a012-18957b5b5d3b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:04.996084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aba7be1b-b932-4809-a012-18957b5b5d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:05.015909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.017234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.017715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.018199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.020642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:05.032405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:05.034596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.036261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:05.055379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:05.056378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:05.057289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:05.057289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:05.057289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:05.061253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 344/688] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:05.061871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.073131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 345/689] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:05.073828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7be1b-b932-4809-a012-18957b5b5d3b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.199289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7be1b-b932-4809-a012-18957b5b5d3b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.202808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03191de-6a90-4b13-bc13-0dadf97dbf95 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.237176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.237538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.237812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.238080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.239131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:05.239445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.240804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.244321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.245150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.245724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.246193 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.247061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:05.247562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.249101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.250483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 345/690] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:05.251776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.263060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 346/691] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:05.268045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.305554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:05.368829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:05.415440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eef3c7-f954-484f-91e2-37326e1abd64 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.421369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5516-f4bb-44fa-a2de-d69049a41115 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.465869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.466473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.466747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.467035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.471769 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.473107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.474239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.474638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.475377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:05.475775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.477239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.478679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.480970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.490384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 346/692] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:05.490384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:13:05.496621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:05.502397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 347/693] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:05.503410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.586411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:05.606894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd5710-73ec-428b-aaf5-58e2645c6802 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.610800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e396a-3de0-4d0c-9e20-c624e5e14388 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.628127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:05.646523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.647151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.647424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.647691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.648328 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.649715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:05.658554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:05.659540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:05.661141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 347/694] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:05.661959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.665887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.666539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.666819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.667094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.667798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] RBAC: Adding query filter params (interface=public, service_id=4fc74ad06ca443b39abdf8f2eea28035, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:05.668106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.672622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.681433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 348/695] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/endpoints?service_id=4fc74ad06ca443b39abdf8f2eea28035&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:05.682235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.783244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dbb93-51d0-4bd1-a166-106c01cfc9bf None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.805453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109cb51-956a-4190-92f2-236903bd9891 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.821160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.821827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.822215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.822513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.823358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:05.823661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.825081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.839224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.839817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:05.840090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:05.840473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:05.841097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:05.841394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:05.842795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:05.844443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 348/696] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:05.845440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.854394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 349/697] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:05.855298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:05.859190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:05.896521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:05.961536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f670e0-2cc0-4395-ba56-cd7cc413b184 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.966633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a633-5446-4d6c-94c3-a46203f2a359 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:05.998589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec0f52-fa1d-48c1-9a40-7981d5b66e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:05.999553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec0f52-fa1d-48c1-9a40-7981d5b66e43 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.000059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec0f52-fa1d-48c1-9a40-7981d5b66e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.000372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d7f0-8236-48cc-83cf-915b36162337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:06.000747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec0f52-fa1d-48c1-9a40-7981d5b66e43 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.000968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d7f0-8236-48cc-83cf-915b36162337 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.001240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d7f0-8236-48cc-83cf-915b36162337 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.001506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d7f0-8236-48cc-83cf-915b36162337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.002142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec0f52-fa1d-48c1-9a40-7981d5b66e43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.004607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec0f52-fa1d-48c1-9a40-7981d5b66e43 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:06.010739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 349/698] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:13:06.011818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec0f52-fa1d-48c1-9a40-7981d5b66e43 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:06.098641 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621d7f0-8236-48cc-83cf-915b36162337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.100482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621d7f0-8236-48cc-83cf-915b36162337 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:06.104851 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:06.164846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec0f52-fa1d-48c1-9a40-7981d5b66e43 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:06.176592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:06.212870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:06.214506 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.215004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.215473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.216411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:06.217016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.218851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:06.225516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 350/699] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:13:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:06.230039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 350/700] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:13:05 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:13:06.231617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:06.236024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d7f0-8236-48cc-83cf-915b36162337 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:06.353363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17af051-e804-46d3-a52a-6db8486f7f7a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:06.373935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d7f0-8236-48cc-83cf-915b36162337 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:06.377398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:06.390419 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:06.391145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.391488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.391822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.392583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] RBAC: Adding query filter params (interface=public, service_id=19147a8941324d46936fbfcb9550e527, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:06.392952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.394425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:06.400964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 351/701] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Oct 17 10:13:06 2022] GET /identity/v3/endpoints?service_id=19147a8941324d46936fbfcb9550e527&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:06.407802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:06.423585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39026e9-95f6-400d-be45-616d5f698b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:06.424208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39026e9-95f6-400d-be45-616d5f698b2f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.424511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39026e9-95f6-400d-be45-616d5f698b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.424739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39026e9-95f6-400d-be45-616d5f698b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.436267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39026e9-95f6-400d-be45-616d5f698b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.437687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39026e9-95f6-400d-be45-616d5f698b2f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:06.446050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f39026e9-95f6-400d-be45-616d5f698b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:06.454404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:06.455381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:06.456287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:06.456287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:06.456287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:06.456287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 351/702] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:06.457564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a028d8-002d-427a-a7ba-54a11efe2fae None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.457564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a028d8-002d-427a-a7ba-54a11efe2fae None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.457564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a028d8-002d-427a-a7ba-54a11efe2fae None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.459365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 352/703] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:06.464025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a028d8-002d-427a-a7ba-54a11efe2fae None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:06.521716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139e8cb-f12d-419a-9616-cf176de7949e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:06.559219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8ea63-e461-4de1-9652-1da241cbdb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:06.559818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8ea63-e461-4de1-9652-1da241cbdb3d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.560092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8ea63-e461-4de1-9652-1da241cbdb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.560357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8ea63-e461-4de1-9652-1da241cbdb3d None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.560969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8ea63-e461-4de1-9652-1da241cbdb3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.562314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8ea63-e461-4de1-9652-1da241cbdb3d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:06.569906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 352/704] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:13:06 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:13:06.571176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee014-26d9-4d3e-84c2-c8b017dffccd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.571460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee014-26d9-4d3e-84c2-c8b017dffccd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.571734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee014-26d9-4d3e-84c2-c8b017dffccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.579931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a028d8-002d-427a-a7ba-54a11efe2fae None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:06.595209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-125ee014-26d9-4d3e-84c2-c8b017dffccd None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:06.596818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-125ee014-26d9-4d3e-84c2-c8b017dffccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:06.628797 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:06.629942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.630058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.630271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.631184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:06.631541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.633300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:06.652218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 353/705] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:06.657876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:06.685410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:06.694709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 353/706] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:06.780193 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09082e2b-d6af-46d7-b1da-f66e92122c39 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:06.816058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:06.816811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.817922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.817922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.824816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.826261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:13:06.839815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:06.840752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:06.841680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 354/707] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:06.846669 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ee014-26d9-4d3e-84c2-c8b017dffccd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:06.876897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:06.952614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ee014-26d9-4d3e-84c2-c8b017dffccd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:06.992130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:06.993677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:06.994293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:06.994600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:06.994870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:06.995573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:06.995885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:06.997340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:07.009738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 354/708] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:07.016941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:07.087973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:07.115811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3725bb-2dbd-456f-bb45-4dbd3ba417cf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:07.129907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:07.143220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec53f7-1849-46f2-af81-2cc5acc12401 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:07.155283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:07.156041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:07.156411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:07.156789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:07.157463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:07.159074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:07.169955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:07.171221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:07.172112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:07.177669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 355/709] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:13:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:07.178705 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:07.184604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:07.185444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:07.185919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:07.186426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:07.187329 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] RBAC: Adding query filter params (role.id=1c2e3fb06553458f8c24ed50ac46ed5e, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=8b862b2975a14e01a5aec3aa936e9375) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:07.187816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:07.189536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:07.202720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 355/710] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:13:07 2022] GET /identity/v3/role_assignments?user.id=8b862b2975a14e01a5aec3aa936e9375&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=1c2e3fb06553458f8c24ed50ac46ed5e => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:07.297075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6687005-47cd-482d-859a-1b5463022ea3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:07.336743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c1494-4f21-4a6d-9552-4292f873f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:07.337369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c1494-4f21-4a6d-9552-4292f873f8ef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:07.337640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c1494-4f21-4a6d-9552-4292f873f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:07.337906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c1494-4f21-4a6d-9552-4292f873f8ef None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:07.338583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c1494-4f21-4a6d-9552-4292f873f8ef None admin] RBAC: Adding query filter params (name=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:07.338881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c1494-4f21-4a6d-9552-4292f873f8ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:07.340271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c1494-4f21-4a6d-9552-4292f873f8ef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:07.346031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 356/711] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:13:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:07.353556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:07.465878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f1d3-81eb-4ce5-ba40-c83e6b336f19 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:07.501663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672cf6f-de19-4f33-a541-02c14568e69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:07.502269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672cf6f-de19-4f33-a541-02c14568e69c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:07.502577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672cf6f-de19-4f33-a541-02c14568e69c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:07.502845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672cf6f-de19-4f33-a541-02c14568e69c None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:07.503466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672cf6f-de19-4f33-a541-02c14568e69c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:07.503762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672cf6f-de19-4f33-a541-02c14568e69c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:07.505714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672cf6f-de19-4f33-a541-02c14568e69c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:07.510718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 356/712] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:13:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:08.284096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dc126-b45d-4ca9-a2ca-ec0a1669655f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:08.284622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dc126-b45d-4ca9-a2ca-ec0a1669655f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:08.284622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dc126-b45d-4ca9-a2ca-ec0a1669655f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:08.287089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 357/713] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:08.292508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0374ba7-74d4-4ae1-bc22-2abb8b70c142 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:08.292791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0374ba7-74d4-4ae1-bc22-2abb8b70c142 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:08.293064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0374ba7-74d4-4ae1-bc22-2abb8b70c142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:08.312240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b0374ba7-74d4-4ae1-bc22-2abb8b70c142 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:08.313404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b0374ba7-74d4-4ae1-bc22-2abb8b70c142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:08.325398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:08.394610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 357/714] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:08.668008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dc126-b45d-4ca9-a2ca-ec0a1669655f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:08.672081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:08.796311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dc126-b45d-4ca9-a2ca-ec0a1669655f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:08.836232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7350804-1a10-4309-aced-6cd115efbf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:08.837069 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350804-1a10-4309-aced-6cd115efbf1e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:08.837577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350804-1a10-4309-aced-6cd115efbf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:08.838075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350804-1a10-4309-aced-6cd115efbf1e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:08.838710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7350804-1a10-4309-aced-6cd115efbf1e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:08.840913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7350804-1a10-4309-aced-6cd115efbf1e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c7350804-1a10-4309-aced-6cd115efbf1e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:08.852942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:08.854004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:08.855110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 358/715] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:13:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:08.860772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0374ba7-74d4-4ae1-bc22-2abb8b70c142 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:08.920343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f7079-0a82-4869-95e9-f579b994a0a9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:08.920520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f7079-0a82-4869-95e9-f579b994a0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:08.920795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f7079-0a82-4869-95e9-f579b994a0a9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:08.922513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 359/716] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:08.927845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138508-0f0a-4540-8546-50d0c9dd3c66 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:08.928129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138508-0f0a-4540-8546-50d0c9dd3c66 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:08.928469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138508-0f0a-4540-8546-50d0c9dd3c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:08.949318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-34138508-0f0a-4540-8546-50d0c9dd3c66 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:08.950564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-34138508-0f0a-4540-8546-50d0c9dd3c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:08.969616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0374ba7-74d4-4ae1-bc22-2abb8b70c142 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:09.011527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:09.012391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.012767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.013135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.013870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:09.014270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:09.015886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:09.022375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 358/717] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:13:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:09.027754 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:09.047745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:09.051563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:09.062408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 360/718] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:09.179091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d0190-dbc7-41c9-a70b-71a875e98f27 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:09.217018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:09.217637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.217915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.218203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.219386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:09.219841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:09.221339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:09.233696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 359/719] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Oct 17 10:13:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:09.239709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138508-0f0a-4540-8546-50d0c9dd3c66 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:09.354616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138508-0f0a-4540-8546-50d0c9dd3c66 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:09.376053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:09.396824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59631644-f2ce-43f9-8ee6-f94ad29c863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:09.398417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59631644-f2ce-43f9-8ee6-f94ad29c863b None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.398910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59631644-f2ce-43f9-8ee6-f94ad29c863b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.399382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59631644-f2ce-43f9-8ee6-f94ad29c863b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.400266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59631644-f2ce-43f9-8ee6-f94ad29c863b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:09.402779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59631644-f2ce-43f9-8ee6-f94ad29c863b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:09.429719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 361/720] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:13:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:09.473930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:09.509770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17fc9-a706-44c1-8eab-ddfc9096b7fe None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:09.556787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97548068-792f-46c0-a7d7-ea04a56f97fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:09.557483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97548068-792f-46c0-a7d7-ea04a56f97fe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.557761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97548068-792f-46c0-a7d7-ea04a56f97fe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.558028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97548068-792f-46c0-a7d7-ea04a56f97fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.558653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97548068-792f-46c0-a7d7-ea04a56f97fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:09.560113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97548068-792f-46c0-a7d7-ea04a56f97fe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-97548068-792f-46c0-a7d7-ea04a56f97fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:09.571182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:09.572280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:09.573140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 360/721] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:13:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:09.580808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59631644-f2ce-43f9-8ee6-f94ad29c863b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:09.609839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7868f5-175f-4f59-8574-253e77970f2a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.610514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7868f5-175f-4f59-8574-253e77970f2a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.611021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7868f5-175f-4f59-8574-253e77970f2a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.613857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 361/722] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:09.619437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4730f-da67-4e36-8842-5bdd8a87a391 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.619969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4730f-da67-4e36-8842-5bdd8a87a391 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.620464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4730f-da67-4e36-8842-5bdd8a87a391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.639232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f8a4730f-da67-4e36-8842-5bdd8a87a391 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:09.640419 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f8a4730f-da67-4e36-8842-5bdd8a87a391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:09.651988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:09.713838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59631644-f2ce-43f9-8ee6-f94ad29c863b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:09.724879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 362/723] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:09.752045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8c71d-b701-49da-abc5-7549dc2df1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:09.752655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c71d-b701-49da-abc5-7549dc2df1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.752934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c71d-b701-49da-abc5-7549dc2df1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.753204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c71d-b701-49da-abc5-7549dc2df1a1 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.754512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8c71d-b701-49da-abc5-7549dc2df1a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:09.754883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8c71d-b701-49da-abc5-7549dc2df1a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:09.756337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8c71d-b701-49da-abc5-7549dc2df1a1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:09.761620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 362/724] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:13:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:09.767329 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4730f-da67-4e36-8842-5bdd8a87a391 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:09.842671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:09.885047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4730f-da67-4e36-8842-5bdd8a87a391 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:09.921354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:09.921956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.922229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.922525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.923153 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:09.923451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:09.924818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:09.926442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a663ce9-da52-40fc-ade3-5967bc049f84 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.926724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a663ce9-da52-40fc-ade3-5967bc049f84 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.927068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a663ce9-da52-40fc-ade3-5967bc049f84 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.928494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 363/725] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:09.930689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 363/726] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:13:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:09.933754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99917ce-e253-4849-87ea-2632691408a5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:09.934037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99917ce-e253-4849-87ea-2632691408a5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:09.934312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99917ce-e253-4849-87ea-2632691408a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:09.937582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:09.953523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f99917ce-e253-4849-87ea-2632691408a5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:09.954856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f99917ce-e253-4849-87ea-2632691408a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:10.039245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 364/727] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:10.040102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99917ce-e253-4849-87ea-2632691408a5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:10.050211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e5c15-7dde-4a7b-826d-342ec25a77ce None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:10.090800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:10.100947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433a0f4-15c3-4e09-85f2-50943cddd966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:10.101546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433a0f4-15c3-4e09-85f2-50943cddd966 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:10.101819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433a0f4-15c3-4e09-85f2-50943cddd966 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:10.102084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433a0f4-15c3-4e09-85f2-50943cddd966 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:10.102784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433a0f4-15c3-4e09-85f2-50943cddd966 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:10.104149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433a0f4-15c3-4e09-85f2-50943cddd966 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:10.108562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:10.134438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 364/728] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:13:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:13:10.172949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99917ce-e253-4849-87ea-2632691408a5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:10.220350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:10.221042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:10.221318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:10.221582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:10.222168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:10.224552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:10.234384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:10.235481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:10.236388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 365/729] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Oct 17 10:13:10 2022] GET /identity/v3/services/image => generated 86 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:10.243870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433a0f4-15c3-4e09-85f2-50943cddd966 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:10.319221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:10.365254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433a0f4-15c3-4e09-85f2-50943cddd966 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:10.405253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:10.406189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:10.406706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:10.407156 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:10.407983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] RBAC: Adding query filter params (name=image) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:10.408477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:10.410003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:10.416243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 365/730] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:10.423042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:10.452598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38afd268-5359-42f0-8719-82459aef3ad8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:10.463357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:10.480242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:10.500456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448be443-1863-482a-a1c4-cf2154360e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:10.501057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448be443-1863-482a-a1c4-cf2154360e01 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:10.501328 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448be443-1863-482a-a1c4-cf2154360e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:10.501619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448be443-1863-482a-a1c4-cf2154360e01 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:10.502370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448be443-1863-482a-a1c4-cf2154360e01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:10.503719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448be443-1863-482a-a1c4-cf2154360e01 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:10.526539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 366/731] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:13:10 2022] POST /identity/v3/services => generated 242 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:13:10.548370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373f9f4-8f52-4aa1-b624-ec1cc6544d4e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:10.600624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d553168-c4d2-4f80-bd33-3eda431f5a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:10.601622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553168-c4d2-4f80-bd33-3eda431f5a74 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:10.602103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553168-c4d2-4f80-bd33-3eda431f5a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:10.602604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553168-c4d2-4f80-bd33-3eda431f5a74 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:10.603443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d553168-c4d2-4f80-bd33-3eda431f5a74 None admin] RBAC: Adding query filter params (type=image) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:10.603947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d553168-c4d2-4f80-bd33-3eda431f5a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:10.605555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d553168-c4d2-4f80-bd33-3eda431f5a74 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:10.611485 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 366/732] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:11.721403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cfb1a-b548-4317-8389-7db096be3216 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:11.721952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cfb1a-b548-4317-8389-7db096be3216 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:11.721952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cfb1a-b548-4317-8389-7db096be3216 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:11.724941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 367/733] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:11.730678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fdc95-6aea-4a37-ac8e-d3a8d6dd7f36 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:11.731500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fdc95-6aea-4a37-ac8e-d3a8d6dd7f36 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:11.732006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fdc95-6aea-4a37-ac8e-d3a8d6dd7f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:11.752399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0b7fdc95-6aea-4a37-ac8e-d3a8d6dd7f36 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:11.754474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b7fdc95-6aea-4a37-ac8e-d3a8d6dd7f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:11.829926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 367/734] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:12.094289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cfb1a-b548-4317-8389-7db096be3216 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:12.201184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cfb1a-b548-4317-8389-7db096be3216 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:12.237460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:12.239037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.239513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.239971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.240813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:12.242476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:12.245346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING keystone.server.flask.common [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 10:13:12.263928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 10:13:12.263928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 10:13:12.263928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [parameters: {'id': '56c5309bd513443b9a7de62f65a45171', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 10:13:12.263928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-236660f3-7797-4414-b623-278b997b15f9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 56c5309bd513443b9a7de62f65a45171.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 56c5309bd513443b9a7de62f65a45171. Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:13:12.272772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 10:13:12.273614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Oct 17 10:13:12.274681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 10:13:12.275592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 10:13:12.276523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:13:12.277367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'id': '56c5309bd513443b9a7de62f65a45171', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:12.278261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:12.281260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 56c5309bd513443b9a7de62f65a45171. Oct 17 10:13:12.283632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.290464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 368/735] 10.222.0.16 () {60 vars in 1140 bytes} [Mon Oct 17 10:13:12 2022] POST /identity/v3/projects => generated 262 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:12.293554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fdc95-6aea-4a37-ac8e-d3a8d6dd7f36 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:12.342527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:12.406376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fdc95-6aea-4a37-ac8e-d3a8d6dd7f36 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:12.407555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202d14d-21b7-4d59-936f-6b5b71e236c1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.407840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202d14d-21b7-4d59-936f-6b5b71e236c1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.408110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202d14d-21b7-4d59-936f-6b5b71e236c1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.417028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 369/736] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:12.418248 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd936-327d-4d25-8328-d2ef99204108 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.418548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd936-327d-4d25-8328-d2ef99204108 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.418816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd936-327d-4d25-8328-d2ef99204108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.438989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ff1dd936-327d-4d25-8328-d2ef99204108 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:12.440835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ff1dd936-327d-4d25-8328-d2ef99204108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:12.444241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:12.444876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.445153 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.445422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.452303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:12.453766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:13:12.465735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:12.466697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:12.468079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 368/737] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:12.470043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:12.475813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:12.519855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 370/738] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:12.580736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:12.585040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816a4d7-fd34-40a3-a28c-29c18f7c1622 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:12.620688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6b2f-0c83-4d04-958e-13c1ebceb8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:12.621262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6b2f-0c83-4d04-958e-13c1ebceb8ce None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.621539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6b2f-0c83-4d04-958e-13c1ebceb8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.621863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6b2f-0c83-4d04-958e-13c1ebceb8ce None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.622555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a6b2f-0c83-4d04-958e-13c1ebceb8ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:12.622863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a6b2f-0c83-4d04-958e-13c1ebceb8ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:12.624296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a6b2f-0c83-4d04-958e-13c1ebceb8ce None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:12.643673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 369/739] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:13:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:12.772055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae6c7-9fd7-4dff-8472-6ffd6b413bbc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.772612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae6c7-9fd7-4dff-8472-6ffd6b413bbc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.772612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae6c7-9fd7-4dff-8472-6ffd6b413bbc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.774316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 371/740] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:12.780903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6755-a428-4725-ab3f-a412c3215e27 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.781437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6755-a428-4725-ab3f-a412c3215e27 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.781437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6755-a428-4725-ab3f-a412c3215e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.801135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eae6c7-9fd7-4dff-8472-6ffd6b413bbc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:12.801927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-539a6755-a428-4725-ab3f-a412c3215e27 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:12.803160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-539a6755-a428-4725-ab3f-a412c3215e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:12.886295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 370/741] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:12.887537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f761-9c19-4bd1-8d9e-2b043afedb7d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.887820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f761-9c19-4bd1-8d9e-2b043afedb7d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.888092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f761-9c19-4bd1-8d9e-2b043afedb7d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.889557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 371/742] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:12.901263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8445fb3-902b-4193-8aee-c326e850563b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.901811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8445fb3-902b-4193-8aee-c326e850563b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.902896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8445fb3-902b-4193-8aee-c326e850563b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.922796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e8445fb3-902b-4193-8aee-c326e850563b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:12.923964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e8445fb3-902b-4193-8aee-c326e850563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:12.925033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eae6c7-9fd7-4dff-8472-6ffd6b413bbc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:12.969019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:12.969647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:12.969923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:12.970187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:12.970816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:12.973701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:12.986490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:12.987747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:12.988864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 372/743] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:13:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:12.995035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.009143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 372/744] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:13.023787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:13.113607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f91e37-bd30-4f1b-bd60-7f411180452f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:13.149104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:13.149624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:13.149894 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:13.150159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:13.150816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:13.151109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:13.152448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:13.157821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 373/745] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:13:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:13.163851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8445fb3-902b-4193-8aee-c326e850563b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.177629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.188507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:13.273546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:13.293912 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d7ce-857f-488f-9982-714415e6b964 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:13.298159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8445fb3-902b-4193-8aee-c326e850563b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:13.330620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:13.331273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:13.331552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:13.331823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:13.332998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:13.334383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:13.336225 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:13.336878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:13.337093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:13.337357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:13.337977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:13.338273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:13.339649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:13.343683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:13.344693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:13.345544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:13.346433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 373/746] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:13:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:13.346510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 374/747] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:13:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:13.346510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.352768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.435696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:13.455783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460f670-d00a-49ce-8ef9-9cadea7979b3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:13.456964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709eb74-3e34-4b46-9ddc-f2a2403514a2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:13.493778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:13.494418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:13.494698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:13.494811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4af05b-6b41-4246-b541-f1d596f22467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:13.495024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:13.495420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4af05b-6b41-4246-b541-f1d596f22467 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:13.495662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] RBAC: Adding query filter params (interface=admin, service_id=19147a8941324d46936fbfcb9550e527, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:13.495757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4af05b-6b41-4246-b541-f1d596f22467 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:13.495946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4af05b-6b41-4246-b541-f1d596f22467 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:13.496040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:13.496598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4af05b-6b41-4246-b541-f1d596f22467 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:13.497308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:13.497942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4af05b-6b41-4246-b541-f1d596f22467 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:13.503014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 374/748] 10.222.0.16 () {56 vars in 1249 bytes} [Mon Oct 17 10:13:13 2022] GET /identity/v3/endpoints?service_id=19147a8941324d46936fbfcb9550e527&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:13.504150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.526546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 375/749] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:13:13 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:13.527993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4af05b-6b41-4246-b541-f1d596f22467 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.540987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:13.645637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c7ab5-81eb-4e79-a339-be7bc6936b98 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:13.669977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4af05b-6b41-4246-b541-f1d596f22467 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:13.698055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:13.699276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:13.700196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:13.700912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:13.701998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] RBAC: Adding query filter params (name=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:13.702905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:13.704825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:13.712391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811d7f7-1fee-4f81-9b0f-a54bc82d7f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:13.712811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d7f7-1fee-4f81-9b0f-a54bc82d7f1c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:13.713080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d7f7-1fee-4f81-9b0f-a54bc82d7f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:13.713347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d7f7-1fee-4f81-9b0f-a54bc82d7f1c None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:13.713956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811d7f7-1fee-4f81-9b0f-a54bc82d7f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:13.715343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 375/750] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:13:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:13.716266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811d7f7-1fee-4f81-9b0f-a54bc82d7f1c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:13.721843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.726900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 376/751] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:13:13 2022] GET /identity/v3/services => generated 1957 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:13:13.742388 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:13.851943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaa4d6-556a-4db3-ab40-5c64e74b11bd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:13.891458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:13.892369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:13.892795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:13.893209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:13.893998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:13.895128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:13.896701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:13.903031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 376/752] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:13:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:13.914453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811d7f7-1fee-4f81-9b0f-a54bc82d7f1c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:13.986380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:14.025760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811d7f7-1fee-4f81-9b0f-a54bc82d7f1c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:14.060697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715341bd-004e-4341-8b27-44753c6cc65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:14.061683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715341bd-004e-4341-8b27-44753c6cc65f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:14.062169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715341bd-004e-4341-8b27-44753c6cc65f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:14.062659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715341bd-004e-4341-8b27-44753c6cc65f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:14.063559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715341bd-004e-4341-8b27-44753c6cc65f None admin] RBAC: Adding query filter params (interface=public, service_id=567b70803a114be296fc92d6a79caf6b, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:14.064064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715341bd-004e-4341-8b27-44753c6cc65f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:14.065786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715341bd-004e-4341-8b27-44753c6cc65f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:14.081642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 377/753] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Oct 17 10:13:13 2022] GET /identity/v3/endpoints?service_id=567b70803a114be296fc92d6a79caf6b&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:14.088094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:14.167895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:14.195492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43d3c2-02ec-414b-a42a-059dfce2117d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:14.230078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff29ec-aff4-40cd-abc0-3997ebcaef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:14.231051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff29ec-aff4-40cd-abc0-3997ebcaef08 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:14.231544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff29ec-aff4-40cd-abc0-3997ebcaef08 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:14.232016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff29ec-aff4-40cd-abc0-3997ebcaef08 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:14.232841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff29ec-aff4-40cd-abc0-3997ebcaef08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:14.234452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff29ec-aff4-40cd-abc0-3997ebcaef08 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:14.241889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 377/754] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:13:14 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:13:14.993303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f71f6f-cf5a-4326-b698-5d9c3040821a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:14.993863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f71f6f-cf5a-4326-b698-5d9c3040821a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:14.993863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f71f6f-cf5a-4326-b698-5d9c3040821a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:14.995440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 378/755] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:15.000795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc9bb0-b436-43d7-8f62-1f41c4e2b65c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:15.001206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc9bb0-b436-43d7-8f62-1f41c4e2b65c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:15.001378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc9bb0-b436-43d7-8f62-1f41c4e2b65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:15.020242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c8fc9bb0-b436-43d7-8f62-1f41c4e2b65c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:15.021656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c8fc9bb0-b436-43d7-8f62-1f41c4e2b65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:15.091272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 378/756] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:14 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:15.366922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f71f6f-cf5a-4326-b698-5d9c3040821a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:15.480603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f71f6f-cf5a-4326-b698-5d9c3040821a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:15.511083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:15.518448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:15.519430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:15.520407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:15.520875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:15.521756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:15.523407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:15.539014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 10:13:15.539014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 10:13:15.539014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [parameters: {'user_id': '7f20ff35daf54da58aeb37874e34a230', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 10:13:15.539014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Oct 17 10:13:15.547113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:13:15.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 10:13:15.548965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 10:13:15.549853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:13:15.551226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:13:15.552096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'user_id': '7f20ff35daf54da58aeb37874e34a230', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:15.552933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 17 10:13:15.553748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 17 10:13:15.556490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 17 10:13:15.556490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 17 10:13:15.556490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 10:13:15.556490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 10:13:15.556490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 10:13:15.556490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:15.557194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 379/757] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Oct 17 10:13:15 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:15.562964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc9bb0-b436-43d7-8f62-1f41c4e2b65c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:15.680214 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc9bb0-b436-43d7-8f62-1f41c4e2b65c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:15.682778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:15.720906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3df495-b0b7-4ce0-99c6-a92706063a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:15.721683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3df495-b0b7-4ce0-99c6-a92706063a4b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:15.721965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3df495-b0b7-4ce0-99c6-a92706063a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:15.722236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3df495-b0b7-4ce0-99c6-a92706063a4b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:15.731553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3df495-b0b7-4ce0-99c6-a92706063a4b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:15.733023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3df495-b0b7-4ce0-99c6-a92706063a4b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ac3df495-b0b7-4ce0-99c6-a92706063a4b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:13:15.745851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:15.748066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:15.750354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 379/758] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:13:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:15.755773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:15.854238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a46c9-a7d2-4a3b-9fb7-78577e96abb1 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:15.854637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a46c9-a7d2-4a3b-9fb7-78577e96abb1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:15.854984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a46c9-a7d2-4a3b-9fb7-78577e96abb1 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:15.856542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 380/759] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:15.862349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c764275-5e0e-4864-bdf4-adae4bdc5d5c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:15.862883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c764275-5e0e-4864-bdf4-adae4bdc5d5c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:15.864002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c764275-5e0e-4864-bdf4-adae4bdc5d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:15.868183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587af985-3e9e-4f35-acbd-9e295e682cee None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:15.883959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2c764275-5e0e-4864-bdf4-adae4bdc5d5c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:15.885492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2c764275-5e0e-4864-bdf4-adae4bdc5d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:15.905278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ab9ff-39e1-4b26-9cee-494ee5124f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:15.906350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ab9ff-39e1-4b26-9cee-494ee5124f15 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:15.906350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ab9ff-39e1-4b26-9cee-494ee5124f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:15.906617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ab9ff-39e1-4b26-9cee-494ee5124f15 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:15.907312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ab9ff-39e1-4b26-9cee-494ee5124f15 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:15.907614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ab9ff-39e1-4b26-9cee-494ee5124f15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:15.909322 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ab9ff-39e1-4b26-9cee-494ee5124f15 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:15.957504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 380/760] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Oct 17 10:13:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:15.978041 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 381/761] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:15 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:16.026712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035fad1-6f9c-4c2c-8c4f-c921178691ec None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.027270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035fad1-6f9c-4c2c-8c4f-c921178691ec None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.027270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035fad1-6f9c-4c2c-8c4f-c921178691ec None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.028753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 381/762] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:16.036193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ed0b7-be47-49f6-bb30-c33d744e6ec5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.036713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ed0b7-be47-49f6-bb30-c33d744e6ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.036713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ed0b7-be47-49f6-bb30-c33d744e6ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.056995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-664ed0b7-be47-49f6-bb30-c33d744e6ec5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:16.058189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-664ed0b7-be47-49f6-bb30-c33d744e6ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:16.134603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:16.149815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 382/763] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:16 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:16.276312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035fad1-6f9c-4c2c-8c4f-c921178691ec None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:16.331258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:16.385501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035fad1-6f9c-4c2c-8c4f-c921178691ec None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:16.419493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482d063-fffe-4c9f-9103-f8bbd98fb696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:16.420115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d063-fffe-4c9f-9103-f8bbd98fb696 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.420394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d063-fffe-4c9f-9103-f8bbd98fb696 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.420661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d063-fffe-4c9f-9103-f8bbd98fb696 None admin] PATH_INFO: `/v3/services/image` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.421244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482d063-fffe-4c9f-9103-f8bbd98fb696 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:16.422641 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482d063-fffe-4c9f-9103-f8bbd98fb696 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:16.429617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ed0b7-be47-49f6-bb30-c33d744e6ec5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7482d063-fffe-4c9f-9103-f8bbd98fb696 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:16.432329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:16.433311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:16.434597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 382/764] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Oct 17 10:13:16 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:16.435401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad52c14-b178-437a-aca5-8350f3cb3df4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.435401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad52c14-b178-437a-aca5-8350f3cb3df4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.435401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad52c14-b178-437a-aca5-8350f3cb3df4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.436741 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 383/765] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:16.439200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad52c14-b178-437a-aca5-8350f3cb3df4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:16.529539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ed0b7-be47-49f6-bb30-c33d744e6ec5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:16.540145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad52c14-b178-437a-aca5-8350f3cb3df4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:16.554140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:16.563062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522ff82-c5c5-433f-8ffc-f34aadbb3fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:16.563677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522ff82-c5c5-433f-8ffc-f34aadbb3fce None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.563952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522ff82-c5c5-433f-8ffc-f34aadbb3fce None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.564214 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522ff82-c5c5-433f-8ffc-f34aadbb3fce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.564791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522ff82-c5c5-433f-8ffc-f34aadbb3fce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:16.566170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522ff82-c5c5-433f-8ffc-f34aadbb3fce None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:16.575297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b522ff82-c5c5-433f-8ffc-f34aadbb3fce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:16.575456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:16.576528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 10:13:16.577401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:16.578151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.578151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.578286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 383/766] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Oct 17 10:13:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:16.578386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.578386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:16.578386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:16.578582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a466e-2d7e-4ae6-864a-15c2fe21c9eb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.578582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a466e-2d7e-4ae6-864a-15c2fe21c9eb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.578582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a466e-2d7e-4ae6-864a-15c2fe21c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.578771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:16.585668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 384/767] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:16.586477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:16.596236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-748a466e-2d7e-4ae6-864a-15c2fe21c9eb None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:16.597424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-748a466e-2d7e-4ae6-864a-15c2fe21c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:16.644005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:16.665047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 384/768] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:16 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:16.666070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a466e-2d7e-4ae6-864a-15c2fe21c9eb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:16.689042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb58cf-7c55-41b1-9d10-2dae09189bc5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:16.727402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:16.728038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.728314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.728580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.729207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:16.729509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:16.730901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:16.735942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 385/769] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:13:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:16.741990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:16.777325 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a466e-2d7e-4ae6-864a-15c2fe21c9eb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:16.788595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:16.814055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:16.814707 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.814984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.815257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.815882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] RBAC: Adding query filter params (type=image) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:16.816778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:16.818130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:16.822927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 385/770] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:16.828620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:16.854059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76780c50-bfa7-4b95-afa6-44609fa65597 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:16.875112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:16.892016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:16.892616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.892892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.893162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.893782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:16.894187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:16.895838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:16.905739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 386/771] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Oct 17 10:13:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:16.906715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:16.932573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f62cf1-df42-4b9c-97ae-33e0d9e5d668 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:16.965533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:16.966151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:16.966455 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:16.966725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:16.967410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] RBAC: Adding query filter params (interface=public, service_id=a30811bad41c4d37885c7d45077f3ce0, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:16.967713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:16.969073 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:16.979199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 386/772] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Oct 17 10:13:16 2022] GET /identity/v3/endpoints?service_id=a30811bad41c4d37885c7d45077f3ce0&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:16.979199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:17.007531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c8987-e5e9-4d21-b691-0891d75d17fc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:17.014230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:17.042899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f464-eca4-4bc7-9bd1-7a89aa70a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:17.043519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f464-eca4-4bc7-9bd1-7a89aa70a8d6 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:17.043696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f464-eca4-4bc7-9bd1-7a89aa70a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:17.043955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f464-eca4-4bc7-9bd1-7a89aa70a8d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:17.044613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f464-eca4-4bc7-9bd1-7a89aa70a8d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:17.045980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f464-eca4-4bc7-9bd1-7a89aa70a8d6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:17.063718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 387/773] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:13:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:13:17.064525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f464-eca4-4bc7-9bd1-7a89aa70a8d6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:17.077528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b5178-286b-4201-9d48-00cecf07bfa3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:17.089372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:17.128882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:17.129834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:17.130293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:17.130756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:17.131494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:17.133079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:17.143677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:17.145019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 10:13:17.146035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:17.148609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 387/774] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:13:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:17.154619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:17.209360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f464-eca4-4bc7-9bd1-7a89aa70a8d6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:17.251955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfbc11-b4a0-4288-b8ae-e68d7fc6078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:17.252720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfbc11-b4a0-4288-b8ae-e68d7fc6078a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:17.252949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfbc11-b4a0-4288-b8ae-e68d7fc6078a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:17.253216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfbc11-b4a0-4288-b8ae-e68d7fc6078a None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:17.253854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfbc11-b4a0-4288-b8ae-e68d7fc6078a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:17.255329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfbc11-b4a0-4288-b8ae-e68d7fc6078a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:17.263106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 388/775] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:13:17 2022] GET /identity/v3/services => generated 1957 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:13:17.265775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcace722-f924-4666-b877-f5b1b209b34c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:17.306096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:17.306668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:17.306938 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:17.307203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:17.307835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] RBAC: Adding query filter params (name=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:17.308130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:17.309491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:17.315193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 388/776] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:13:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:17.321294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfbc11-b4a0-4288-b8ae-e68d7fc6078a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:17.355312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:17.435734 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfbc11-b4a0-4288-b8ae-e68d7fc6078a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:17.475360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c40ba-a7e1-4c72-9fc0-3235990b11c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:17.475967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c40ba-a7e1-4c72-9fc0-3235990b11c1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:17.476239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c40ba-a7e1-4c72-9fc0-3235990b11c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:17.476504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c40ba-a7e1-4c72-9fc0-3235990b11c1 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:17.477129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c40ba-a7e1-4c72-9fc0-3235990b11c1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:17.477427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c40ba-a7e1-4c72-9fc0-3235990b11c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:17.478830 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c40ba-a7e1-4c72-9fc0-3235990b11c1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:17.485004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 389/777] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Oct 17 10:13:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:17.489979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:17.525923 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:17.595322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a16ac-3261-49d7-aceb-ef36b331f5de None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:17.630148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dc0c8-710f-446c-9645-d25c6c3e4905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:17.630785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc0c8-710f-446c-9645-d25c6c3e4905 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:17.631064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc0c8-710f-446c-9645-d25c6c3e4905 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:17.631330 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc0c8-710f-446c-9645-d25c6c3e4905 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:17.631985 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dc0c8-710f-446c-9645-d25c6c3e4905 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:17.633354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dc0c8-710f-446c-9645-d25c6c3e4905 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:17.661452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 389/778] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:13:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:19.255889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f28d-1d05-43f1-88de-fca0d2fe6c04 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.256414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f28d-1d05-43f1-88de-fca0d2fe6c04 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.256414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f28d-1d05-43f1-88de-fca0d2fe6c04 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.257969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 390/779] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:19.263087 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b5e19-7693-4010-914b-cefcfb624868 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.263482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b5e19-7693-4010-914b-cefcfb624868 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.263671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b5e19-7693-4010-914b-cefcfb624868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.281830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-167b5e19-7693-4010-914b-cefcfb624868 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:19.283019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-167b5e19-7693-4010-914b-cefcfb624868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:19.362149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 390/780] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:19.444903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fd1d5-e3da-4d1e-9c61-7c2b1d4d10b5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.445222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fd1d5-e3da-4d1e-9c61-7c2b1d4d10b5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.445449 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fd1d5-e3da-4d1e-9c61-7c2b1d4d10b5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.446969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 391/781] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:19.451979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e2a79-8450-4d53-bfa5-25a236372b84 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.452258 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e2a79-8450-4d53-bfa5-25a236372b84 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.452524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e2a79-8450-4d53-bfa5-25a236372b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.471656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1d1e2a79-8450-4d53-bfa5-25a236372b84 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:19.474044 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1d1e2a79-8450-4d53-bfa5-25a236372b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:19.484825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:19.544124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 391/782] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:19.627330 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fd1d5-e3da-4d1e-9c61-7c2b1d4d10b5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:19.720962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:19.731170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fd1d5-e3da-4d1e-9c61-7c2b1d4d10b5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:19.766201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a753d62-cc89-47a7-a208-9dc7694d19e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:19.766887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a753d62-cc89-47a7-a208-9dc7694d19e4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.767159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a753d62-cc89-47a7-a208-9dc7694d19e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.767429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a753d62-cc89-47a7-a208-9dc7694d19e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.768621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a753d62-cc89-47a7-a208-9dc7694d19e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:19.770144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a753d62-cc89-47a7-a208-9dc7694d19e4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:19.772052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 392/783] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:19.777851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e2a79-8450-4d53-bfa5-25a236372b84 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:19.809000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a753d62-cc89-47a7-a208-9dc7694d19e4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:19.881909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e2a79-8450-4d53-bfa5-25a236372b84 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:19.913839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a753d62-cc89-47a7-a208-9dc7694d19e4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:19.915376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15556a-b519-42d1-b6e6-fa75d35119ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:19.915975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15556a-b519-42d1-b6e6-fa75d35119ad None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.916269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15556a-b519-42d1-b6e6-fa75d35119ad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.916536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15556a-b519-42d1-b6e6-fa75d35119ad None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.917170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15556a-b519-42d1-b6e6-fa75d35119ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:19.918617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15556a-b519-42d1-b6e6-fa75d35119ad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:19.938789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:19.946772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:19.947174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 392/784] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:13:19 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:19.947392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.947664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.947927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] PATH_INFO: `/v3/services/image` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.948505 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:19.948602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ec9a7-9346-4c62-ac4d-eda2fafdc7b7 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.948808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ec9a7-9346-4c62-ac4d-eda2fafdc7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.949055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ec9a7-9346-4c62-ac4d-eda2fafdc7b7 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.949988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:19.950465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 393/785] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:19.956528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a58539-0d8a-4ae4-abe8-4f80cc4ffe63 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:19.956858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a58539-0d8a-4ae4-abe8-4f80cc4ffe63 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:19.957136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a58539-0d8a-4ae4-abe8-4f80cc4ffe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:19.969120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:19.970258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 10:13:19.971703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:19.974520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 393/786] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Oct 17 10:13:19 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:19.977510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:19.981423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-57a58539-0d8a-4ae4-abe8-4f80cc4ffe63 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:19.982826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-57a58539-0d8a-4ae4-abe8-4f80cc4ffe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:20.009933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:20.059659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 394/787] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:20.102178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070576b-7b88-4c12-acb9-b5b0e0b83c59 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:20.140098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:20.140619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:20.140892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:20.141157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:20.141775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] RBAC: Adding query filter params (name=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:20.142072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:20.143456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:20.149082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 394/788] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:20.154871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a58539-0d8a-4ae4-abe8-4f80cc4ffe63 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:20.256020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a58539-0d8a-4ae4-abe8-4f80cc4ffe63 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:20.282853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:20.292777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:20.293378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:20.293651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:20.293918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:20.294563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] RBAC: Adding query filter params (type=image) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:20.294864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:20.296217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:20.301365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 395/789] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:20.306833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:20.338551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:20.404690 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:20.417043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cbb00-d893-49a5-939f-dd5b2f67568b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:20.447124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef3a3e-6973-4a8d-a75a-363a77c1a8fd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:20.454704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae88e2-554d-4484-a4ce-da91cffeb4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:20.455300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae88e2-554d-4484-a4ce-da91cffeb4a8 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:20.455572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae88e2-554d-4484-a4ce-da91cffeb4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:20.455833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae88e2-554d-4484-a4ce-da91cffeb4a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:20.456498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae88e2-554d-4484-a4ce-da91cffeb4a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:20.457847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae88e2-554d-4484-a4ce-da91cffeb4a8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:20.480541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 395/790] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:13:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:20.497505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75c65-d18d-4e7e-8b9b-42a07bb21549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:20.498161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75c65-d18d-4e7e-8b9b-42a07bb21549 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:20.498481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75c65-d18d-4e7e-8b9b-42a07bb21549 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:20.498755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75c65-d18d-4e7e-8b9b-42a07bb21549 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:20.512546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab75c65-d18d-4e7e-8b9b-42a07bb21549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6ab75c65-d18d-4e7e-8b9b-42a07bb21549 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:13:20.517558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:20.518934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 10:13:20.519957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:20.521316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 396/791] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Oct 17 10:13:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:20.525256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae88e2-554d-4484-a4ce-da91cffeb4a8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:20.635962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae88e2-554d-4484-a4ce-da91cffeb4a8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:20.673702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:20.674231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:20.674513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:20.674782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:20.675473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:20.676406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:20.677980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:20.686188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 396/792] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Oct 17 10:13:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:20.693832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75c65-d18d-4e7e-8b9b-42a07bb21549 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:20.744458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:20.798284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75c65-d18d-4e7e-8b9b-42a07bb21549 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:20.831838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:20.832718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:20.832993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:20.833257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:20.843891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:20.845329 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:20.860190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:20.861173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:20.862094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:20.862094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:20.862094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:20.862094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 397/793] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:20.868492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:20.880472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:20.973159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b59d6-290d-438c-b0a8-75311f5e2105 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:21.006569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:21.007166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:21.007445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:21.007711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:21.008346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:21.008644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:21.009984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:21.027124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 397/794] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:21.028464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:21.133289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:21.136884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c311d-5b34-47e3-87af-f8fe961f9da1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:21.171280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:21.172062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:21.172433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:21.172799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:21.182163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:21.183728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:13:21.194959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:21.196125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:13:21.197323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:21.198888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 398/795] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Oct 17 10:13:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:21.204421 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:21.273858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:21.308072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae015c4-3af4-4d6d-8053-6ae5d5138895 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:21.342435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:21.343030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:21.343299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:21.343564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:21.344272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:21.344572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:21.345968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:21.363475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 398/796] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Oct 17 10:13:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:21.369129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:21.470690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3a689-0a55-4ac8-b8d2-7826508e91fb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:21.504764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:21.505379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:21.505653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:21.505921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:21.516322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:21.517756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:21.531742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:21.532651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:21.533556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:21.533556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:21.533556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:21.534271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 399/797] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:21.539019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:21.638650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904a7d5-12c1-46fb-9a98-25203d9ad3d1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:21.663838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:21.673697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:21.674388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:21.674725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:21.675056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:21.676374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:21.676740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:21.678137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:21.694052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 399/798] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:21.695250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:21.710604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:21.801619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda1d6d-5a12-4a8e-bfa2-67ebe96e0da1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:21.835639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:21.836394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:21.836764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:21.837126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:21.843711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:21.845291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:13:21.857057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:21.858158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:21.859327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:13:21.859327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:13:21.859327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:13:21.859327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:13:21.859327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:13:21.859327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:13:21.859327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:21.859327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:21.864934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 400/799] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:13:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:21.867564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:21.963026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a13d4b-35d9-4016-8a67-fc6ecaa9fcd1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:21.995135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18020126-fa8a-4d1c-98e3-1ee2333bcea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:21.995725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18020126-fa8a-4d1c-98e3-1ee2333bcea9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:21.995994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18020126-fa8a-4d1c-98e3-1ee2333bcea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:21.996259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18020126-fa8a-4d1c-98e3-1ee2333bcea9 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:21.996934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18020126-fa8a-4d1c-98e3-1ee2333bcea9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:21.997230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18020126-fa8a-4d1c-98e3-1ee2333bcea9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:21.998683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18020126-fa8a-4d1c-98e3-1ee2333bcea9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:22.010886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 400/800] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:22.016960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:22.091471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:22.124674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3162f3-cb82-4f13-936c-6d81e1ca9ae4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:22.159143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06926e90-2ffb-4304-bcf4-018e308a4c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:22.160304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06926e90-2ffb-4304-bcf4-018e308a4c52 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:22.160587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06926e90-2ffb-4304-bcf4-018e308a4c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:22.160855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06926e90-2ffb-4304-bcf4-018e308a4c52 None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/9d52624718de4457adc44e9965cd7a09/roles/1c2e3fb06553458f8c24ed50ac46ed5e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:22.163265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06926e90-2ffb-4304-bcf4-018e308a4c52 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=9d52624718de4457adc44e9965cd7a09, role_id=1c2e3fb06553458f8c24ed50ac46ed5e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:22.164698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06926e90-2ffb-4304-bcf4-018e308a4c52 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:22.178991 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 401/801] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:13:22 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/9d52624718de4457adc44e9965cd7a09/roles/1c2e3fb06553458f8c24ed50ac46ed5e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:13:22.192499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e73db-9acf-4874-a07a-65d8c1527597 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:22.192834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e73db-9acf-4874-a07a-65d8c1527597 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:22.193053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e73db-9acf-4874-a07a-65d8c1527597 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:22.194786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 401/802] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:22.200681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee34107-fbb2-4a8c-871f-d131db53e56f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:22.200975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee34107-fbb2-4a8c-871f-d131db53e56f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:22.201236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee34107-fbb2-4a8c-871f-d131db53e56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:22.338649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3ee34107-fbb2-4a8c-871f-d131db53e56f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:22.339777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3ee34107-fbb2-4a8c-871f-d131db53e56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:22.413676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 402/803] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:22.685791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e73db-9acf-4874-a07a-65d8c1527597 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:22.729639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7797d-36a6-493b-92d4-0c94e90b0b9c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:22.729942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7797d-36a6-493b-92d4-0c94e90b0b9c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:22.730489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7797d-36a6-493b-92d4-0c94e90b0b9c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:22.732045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 403/804] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:22.736038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:22.736829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc2617-2479-4bf2-84a7-d8c16434c3c5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:22.737152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc2617-2479-4bf2-84a7-d8c16434c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:22.737373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc2617-2479-4bf2-84a7-d8c16434c3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:22.755912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-60dc2617-2479-4bf2-84a7-d8c16434c3c5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:22.757299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-60dc2617-2479-4bf2-84a7-d8c16434c3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:22.796150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e73db-9acf-4874-a07a-65d8c1527597 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:22.817034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:22.830669 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 404/805] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:22.836830 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e22d62-7a23-4f08-b409-3478116ad2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:22.837451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22d62-7a23-4f08-b409-3478116ad2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:22.837720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22d62-7a23-4f08-b409-3478116ad2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:22.837982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22d62-7a23-4f08-b409-3478116ad2b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:22.839087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e22d62-7a23-4f08-b409-3478116ad2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:22.840493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e22d62-7a23-4f08-b409-3478116ad2b5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:22.842356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 402/806] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:22.848726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc2617-2479-4bf2-84a7-d8c16434c3c5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:22.950716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc2617-2479-4bf2-84a7-d8c16434c3c5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:22.984728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe89f51-f033-4737-b156-a69ff1d3502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:22.985386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe89f51-f033-4737-b156-a69ff1d3502e None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:22.985661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe89f51-f033-4737-b156-a69ff1d3502e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:22.985929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe89f51-f033-4737-b156-a69ff1d3502e None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:22.986683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe89f51-f033-4737-b156-a69ff1d3502e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:22.988729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe89f51-f033-4737-b156-a69ff1d3502e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:23.016289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 405/807] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:13:22 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:23.106484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e22d62-7a23-4f08-b409-3478116ad2b5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:23.212777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e22d62-7a23-4f08-b409-3478116ad2b5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:23.228306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:23.250591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:23.251183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:23.251453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:23.251714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:23.252365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:23.254303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:23.262009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 10:13:23.262009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 10:13:23.262009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [parameters: {'id': '55afcb71e7c94f70a84834c20545a485', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 10:13:23.262009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:13:23.269651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 10:13:23.270818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 10:13:23.271864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 10:13:23.272787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 10:13:23.273701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 10:13:23.274605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'id': '55afcb71e7c94f70a84834c20545a485', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 10:13:23.275487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:23.276321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:23.277244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 403/808] 10.222.0.16 () {60 vars in 1137 bytes} [Mon Oct 17 10:13:23 2022] POST /identity/v3/domains => generated 190 bytes in 166 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:23.280180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe89f51-f033-4737-b156-a69ff1d3502e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:23.334085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:23.389305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe89f51-f033-4737-b156-a69ff1d3502e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:23.424364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e82c6-b2f1-466b-bbf2-0da394318ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:23.425033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e82c6-b2f1-466b-bbf2-0da394318ebe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:23.425249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e82c6-b2f1-466b-bbf2-0da394318ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:23.426068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e82c6-b2f1-466b-bbf2-0da394318ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:23.436915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e82c6-b2f1-466b-bbf2-0da394318ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:23.438360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e82c6-b2f1-466b-bbf2-0da394318ebe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-010e82c6-b2f1-466b-bbf2-0da394318ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:23.453935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:23.455084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:23.456042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:23.456042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:23.456042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:23.458405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 406/809] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:23.461691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:23.569861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e86908-d1d7-4bbd-8702-256c0e96c4f7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:23.604784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcf9db-7d4b-4c64-9620-edca0a4306df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:23.605379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf9db-7d4b-4c64-9620-edca0a4306df None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:23.605647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf9db-7d4b-4c64-9620-edca0a4306df None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:23.605909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf9db-7d4b-4c64-9620-edca0a4306df None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:23.606565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcf9db-7d4b-4c64-9620-edca0a4306df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:23.606860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcf9db-7d4b-4c64-9620-edca0a4306df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:23.608192 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcf9db-7d4b-4c64-9620-edca0a4306df None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:23.619994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 404/810] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:24.357105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5cc5-2f8b-41f4-b278-d1acb01dc8c2 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:24.357307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5cc5-2f8b-41f4-b278-d1acb01dc8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:24.357566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5cc5-2f8b-41f4-b278-d1acb01dc8c2 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:24.358931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 407/811] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:24.364881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4608-be54-4ddb-9af2-c823f8b8f5f4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:24.365378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4608-be54-4ddb-9af2-c823f8b8f5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:24.365472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4608-be54-4ddb-9af2-c823f8b8f5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:24.384039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ceef4608-be54-4ddb-9af2-c823f8b8f5f4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:24.385639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ceef4608-be54-4ddb-9af2-c823f8b8f5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:24.414498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:24.451589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 405/812] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:24.717213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea5cc5-2f8b-41f4-b278-d1acb01dc8c2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:24.814657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:24.819098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea5cc5-2f8b-41f4-b278-d1acb01dc8c2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:24.853033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201b15d-a077-4b04-b4e1-b01befbfa221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:24.853655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b15d-a077-4b04-b4e1-b01befbfa221 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:24.853934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b15d-a077-4b04-b4e1-b01befbfa221 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:24.854199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b15d-a077-4b04-b4e1-b01befbfa221 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:24.864521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201b15d-a077-4b04-b4e1-b01befbfa221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-4201b15d-a077-4b04-b4e1-b01befbfa221 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:13:24.869213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:24.870186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:24.871134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 408/813] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Oct 17 10:13:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:24.876300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef4608-be54-4ddb-9af2-c823f8b8f5f4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:24.971829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef4608-be54-4ddb-9af2-c823f8b8f5f4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:25.005781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:25.006398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.006670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.006934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.007584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:25.007877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:25.009210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:25.016774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 406/814] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Oct 17 10:13:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:25.023585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201b15d-a077-4b04-b4e1-b01befbfa221 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:25.129123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201b15d-a077-4b04-b4e1-b01befbfa221 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:25.163877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32413634-e72d-4610-bb64-6939a110504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:25.164496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32413634-e72d-4610-bb64-6939a110504b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.164770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32413634-e72d-4610-bb64-6939a110504b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.165038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32413634-e72d-4610-bb64-6939a110504b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.174743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32413634-e72d-4610-bb64-6939a110504b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:25.176375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32413634-e72d-4610-bb64-6939a110504b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-32413634-e72d-4610-bb64-6939a110504b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:25.190372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:25.192065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:25.192964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:25.192964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:25.192964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:25.192964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 409/815] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:25.197833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:25.228397 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a203e-d8d2-44ff-9fd0-1e2cbe85bded None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.228588 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a203e-d8d2-44ff-9fd0-1e2cbe85bded None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.228861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a203e-d8d2-44ff-9fd0-1e2cbe85bded None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.230179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 410/816] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:13:25.230307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:25.235244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4a1e0-bf8f-436a-95f3-a2c3cd0478f3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.235516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4a1e0-bf8f-436a-95f3-a2c3cd0478f3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.235779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4a1e0-bf8f-436a-95f3-a2c3cd0478f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.254232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8cd4a1e0-bf8f-436a-95f3-a2c3cd0478f3 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:25.255874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8cd4a1e0-bf8f-436a-95f3-a2c3cd0478f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:25.263242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:25.299113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d284578-3e3c-4ece-8387-afad44064e4b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:25.323144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 411/817] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:13:25.333176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:25.333765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.334040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.334347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.334983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:25.335273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:25.336603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:25.348129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 407/818] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:25.353564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4a1e0-bf8f-436a-95f3-a2c3cd0478f3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:25.447170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4a1e0-bf8f-436a-95f3-a2c3cd0478f3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:25.480217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:25.480759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.481027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.481287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.488556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:25.490055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:13:25.501232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:25.502412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:25.504354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 412/819] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:25.508635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:25.594798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:25.598421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:25.601233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:25.605886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe45807-c955-42a5-a088-d8a76aa62813 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:25.640131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:25.640722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.641025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.641291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.642006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:25.642299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:25.643732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:25.660422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 408/820] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:25.666133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:25.698005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48910c9-fb63-4be4-8fe6-5a64b15ab4ad None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:25.731496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ac4b-48ab-4c27-88d4-71a87efe4035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:25.732106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ac4b-48ab-4c27-88d4-71a87efe4035 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.732360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ac4b-48ab-4c27-88d4-71a87efe4035 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.732624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ac4b-48ab-4c27-88d4-71a87efe4035 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.743759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6ac4b-48ab-4c27-88d4-71a87efe4035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-71f6ac4b-48ab-4c27-88d4-71a87efe4035 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:13:25.748486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:25.749517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:25.750422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 413/821] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:25.756262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ac4b-48ab-4c27-88d4-71a87efe4035 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:25.766197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5153dd-837e-49d3-b279-2d7977ce7fef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:25.798841 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94f0e-11f2-4a6e-8951-a465dfef43a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:25.799431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94f0e-11f2-4a6e-8951-a465dfef43a2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.799702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94f0e-11f2-4a6e-8951-a465dfef43a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.799968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94f0e-11f2-4a6e-8951-a465dfef43a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.815109 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:25.818075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:25.854994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ac4b-48ab-4c27-88d4-71a87efe4035 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:25.867236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e94f0e-11f2-4a6e-8951-a465dfef43a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:25.868665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e94f0e-11f2-4a6e-8951-a465dfef43a2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:25.888859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:25.889445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:25.889712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:25.889974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:25.890659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:25.890956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:25.892291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:25.900776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 414/822] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:25.906549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:25.968228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 409/823] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 17 10:13:25.974706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94f0e-11f2-4a6e-8951-a465dfef43a2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:26.004239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5e1f2-f37b-4ee6-86fd-bbb68d3298e2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:26.023289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:26.027657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:26.039282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4292e-a956-4553-aa92-e004df66622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:26.039897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4292e-a956-4553-aa92-e004df66622d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:26.040169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4292e-a956-4553-aa92-e004df66622d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:26.040439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4292e-a956-4553-aa92-e004df66622d None admin] PATH_INFO: `/v3/users/9c92d2141de649eba969e78baa760b3a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:26.053782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4292e-a956-4553-aa92-e004df66622d None admin] RBAC: Authorizing `identity:get_user(user_id=9c92d2141de649eba969e78baa760b3a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:26.055235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4292e-a956-4553-aa92-e004df66622d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:26.056880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 415/824] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/users/9c92d2141de649eba969e78baa760b3a => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:26.062568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4292e-a956-4553-aa92-e004df66622d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:26.079623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94f0e-11f2-4a6e-8951-a465dfef43a2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:26.115205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:26.115829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:26.116102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:26.116372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:26.126518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:26.128046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:13:26.142597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:26.143583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:13:26.144611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:13:26.144611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:13:26.144611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:26.144611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 410/825] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:26.149444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:26.161050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4292e-a956-4553-aa92-e004df66622d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:26.195878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86fd36-fdf2-4b99-919a-847932a1d8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:26.196496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86fd36-fdf2-4b99-919a-847932a1d8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:26.196771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86fd36-fdf2-4b99-919a-847932a1d8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:26.197037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86fd36-fdf2-4b99-919a-847932a1d8c5 None admin] PATH_INFO: `/v3/projects/bd56c4e6b5974fb1bd199b0315113281` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:26.198241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86fd36-fdf2-4b99-919a-847932a1d8c5 None admin] RBAC: Authorizing `identity:get_project(project_id=bd56c4e6b5974fb1bd199b0315113281)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:26.199692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86fd36-fdf2-4b99-919a-847932a1d8c5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:26.201290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 416/826] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:13:26 2022] GET /identity/v3/projects/bd56c4e6b5974fb1bd199b0315113281 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:26.207840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86fd36-fdf2-4b99-919a-847932a1d8c5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:26.246488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:26.249532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495f3c4-3c2c-468c-984a-d3f85c21a9d5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:26.258187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:26.284623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:26.285268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:26.285542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:26.285814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:26.286478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:26.286780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:26.288126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:26.299417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 411/827] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:26.305551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:26.307768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86fd36-fdf2-4b99-919a-847932a1d8c5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:26.341677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2641d79-3af0-408d-af75-187aef0276dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:26.342356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2641d79-3af0-408d-af75-187aef0276dd None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:26.342634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2641d79-3af0-408d-af75-187aef0276dd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:26.342903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2641d79-3af0-408d-af75-187aef0276dd None admin] PATH_INFO: `/v3/projects/bd56c4e6b5974fb1bd199b0315113281/users/9c92d2141de649eba969e78baa760b3a/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:26.344782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2641d79-3af0-408d-af75-187aef0276dd None admin] RBAC: Authorizing `identity:create_grant(project_id=bd56c4e6b5974fb1bd199b0315113281, user_id=9c92d2141de649eba969e78baa760b3a, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:26.346203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2641d79-3af0-408d-af75-187aef0276dd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:26.360906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 417/828] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:13:26 2022] PUT /identity/v3/projects/bd56c4e6b5974fb1bd199b0315113281/users/9c92d2141de649eba969e78baa760b3a/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:13:26.409115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfc5c9-4f34-44e0-946a-6b365bc10acc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:26.448262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:26.448893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:26.449162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:26.449422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:26.457348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:26.459473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:13:26.591412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:26.592796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:26.593688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 412/829] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:13:26.599184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2641d79-3af0-408d-af75-187aef0276dd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:26.698468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2641d79-3af0-408d-af75-187aef0276dd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:26.708975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:26.734601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c182-78cd-462b-9398-73a4e6d5ac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:26.735255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c182-78cd-462b-9398-73a4e6d5ac17 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:26.735460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c182-78cd-462b-9398-73a4e6d5ac17 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:26.735726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c182-78cd-462b-9398-73a4e6d5ac17 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:26.736484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2c182-78cd-462b-9398-73a4e6d5ac17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:26.736805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2c182-78cd-462b-9398-73a4e6d5ac17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:26.738471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2c182-78cd-462b-9398-73a4e6d5ac17 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:26.750095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 418/830] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:13:26.756768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:26.778533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:26.854831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f0d3f-c1ee-4036-a16f-4fbc56030ea9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:26.887235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0872a8-cbd2-4bf6-896c-47583942565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:26.887829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0872a8-cbd2-4bf6-896c-47583942565f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:26.888101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0872a8-cbd2-4bf6-896c-47583942565f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:26.888368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0872a8-cbd2-4bf6-896c-47583942565f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:26.889078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0872a8-cbd2-4bf6-896c-47583942565f None admin] RBAC: Adding query filter params (role.id=1c2e3fb06553458f8c24ed50ac46ed5e, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=9d52624718de4457adc44e9965cd7a09) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:26.889378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0872a8-cbd2-4bf6-896c-47583942565f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:26.890824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0872a8-cbd2-4bf6-896c-47583942565f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:26.897176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 413/831] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:13:26 2022] GET /identity/v3/role_assignments?user.id=9d52624718de4457adc44e9965cd7a09&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=1c2e3fb06553458f8c24ed50ac46ed5e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:28.487125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc1bd0-eebc-4700-8443-324c2482edf8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:28.487650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc1bd0-eebc-4700-8443-324c2482edf8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:28.487650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc1bd0-eebc-4700-8443-324c2482edf8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:28.489552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 419/832] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:28.494636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839d628-d7ea-450c-b0e0-0b0d6717497f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:28.495053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839d628-d7ea-450c-b0e0-0b0d6717497f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:28.495161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839d628-d7ea-450c-b0e0-0b0d6717497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:28.512822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f839d628-d7ea-450c-b0e0-0b0d6717497f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:28.513983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f839d628-d7ea-450c-b0e0-0b0d6717497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:28.577676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 414/833] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:28.844119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc1bd0-eebc-4700-8443-324c2482edf8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:28.942958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc1bd0-eebc-4700-8443-324c2482edf8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:28.975888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3054-0fb5-42ea-9e12-89d50612bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:28.976496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3054-0fb5-42ea-9e12-89d50612bad8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:28.976772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3054-0fb5-42ea-9e12-89d50612bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:28.977042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3054-0fb5-42ea-9e12-89d50612bad8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:28.986837 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc3054-0fb5-42ea-9e12-89d50612bad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b6cc3054-0fb5-42ea-9e12-89d50612bad8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:13:28.991592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:28.992683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:28.993699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 420/834] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:13:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:28.998601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839d628-d7ea-450c-b0e0-0b0d6717497f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:29.013830 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:29.094741 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839d628-d7ea-450c-b0e0-0b0d6717497f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:29.128092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:29.128765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:29.129131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:29.129405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:29.130065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:29.130387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:29.131732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:29.139489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 415/835] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:13:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:29.145339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3054-0fb5-42ea-9e12-89d50612bad8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:29.176048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:29.244564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3054-0fb5-42ea-9e12-89d50612bad8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:29.276709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff869-4809-4767-8eb1-3d6cfd52e75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:29.277315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff869-4809-4767-8eb1-3d6cfd52e75f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:29.277589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff869-4809-4767-8eb1-3d6cfd52e75f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:29.277854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff869-4809-4767-8eb1-3d6cfd52e75f None admin] PATH_INFO: `/v3/users/9c92d2141de649eba969e78baa760b3a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:29.278947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ff869-4809-4767-8eb1-3d6cfd52e75f None admin] RBAC: Authorizing `identity:get_user(user_id=9c92d2141de649eba969e78baa760b3a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:29.280349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ff869-4809-4767-8eb1-3d6cfd52e75f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:29.281874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 421/836] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:13:29 2022] GET /identity/v3/users/9c92d2141de649eba969e78baa760b3a => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:29.288344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:29.362076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:29.384406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ebbff-40d9-4c67-aea5-95fd373cdf52 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:29.416490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eabf33-f4b4-473c-808d-e84eb14daee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:29.417068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eabf33-f4b4-473c-808d-e84eb14daee1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:29.417335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eabf33-f4b4-473c-808d-e84eb14daee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:29.417596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eabf33-f4b4-473c-808d-e84eb14daee1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:29.480334 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eabf33-f4b4-473c-808d-e84eb14daee1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:29.481740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eabf33-f4b4-473c-808d-e84eb14daee1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:29.553661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:29.577025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 416/837] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:13:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:13:29.583848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff869-4809-4767-8eb1-3d6cfd52e75f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:29.678710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:29.682530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff869-4809-4767-8eb1-3d6cfd52e75f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:29.715883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a07f5-5a21-4267-8059-9614efae4cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:29.716495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a07f5-5a21-4267-8059-9614efae4cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:29.716770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a07f5-5a21-4267-8059-9614efae4cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:29.717037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a07f5-5a21-4267-8059-9614efae4cf3 None admin] PATH_INFO: `/v3/projects/bd56c4e6b5974fb1bd199b0315113281` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:29.718125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a07f5-5a21-4267-8059-9614efae4cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=bd56c4e6b5974fb1bd199b0315113281)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:29.719726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a07f5-5a21-4267-8059-9614efae4cf3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:29.721613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 422/838] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:13:29 2022] GET /identity/v3/projects/bd56c4e6b5974fb1bd199b0315113281 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:29.728333 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eabf33-f4b4-473c-808d-e84eb14daee1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:29.823924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eabf33-f4b4-473c-808d-e84eb14daee1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:29.856908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240bc1-969c-4cf1-ab65-96fc3bc994cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:29.857476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240bc1-969c-4cf1-ab65-96fc3bc994cf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:29.857743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240bc1-969c-4cf1-ab65-96fc3bc994cf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:29.858004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240bc1-969c-4cf1-ab65-96fc3bc994cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:29.858734 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240bc1-969c-4cf1-ab65-96fc3bc994cf None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=bd56c4e6b5974fb1bd199b0315113281, user.id=9c92d2141de649eba969e78baa760b3a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:29.859030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240bc1-969c-4cf1-ab65-96fc3bc994cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:29.860414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240bc1-969c-4cf1-ab65-96fc3bc994cf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:29.866420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 417/839] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:13:29 2022] GET /identity/v3/role_assignments?user.id=9c92d2141de649eba969e78baa760b3a&scope.project.id=bd56c4e6b5974fb1bd199b0315113281&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:31.976584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f034-5fd9-4366-9931-b8490a7579bc None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:31.977145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f034-5fd9-4366-9931-b8490a7579bc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:31.977145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f034-5fd9-4366-9931-b8490a7579bc None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:31.978490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 423/840] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:31.983536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b278471-c3e1-4f7e-a87c-b0bd9093407a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:31.983932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b278471-c3e1-4f7e-a87c-b0bd9093407a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:31.984068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b278471-c3e1-4f7e-a87c-b0bd9093407a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:32.001694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2b278471-c3e1-4f7e-a87c-b0bd9093407a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:32.002878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2b278471-c3e1-4f7e-a87c-b0bd9093407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:32.035551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:32.067225 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 418/841] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:32.338546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6f034-5fd9-4366-9931-b8490a7579bc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:32.436256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6f034-5fd9-4366-9931-b8490a7579bc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:32.468673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d14598-275e-4a0f-a386-579ad0e36505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:32.469320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14598-275e-4a0f-a386-579ad0e36505 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:32.469599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14598-275e-4a0f-a386-579ad0e36505 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:32.469869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14598-275e-4a0f-a386-579ad0e36505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:32.470981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d14598-275e-4a0f-a386-579ad0e36505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:32.473249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d14598-275e-4a0f-a386-579ad0e36505 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:32.474811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 424/842] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:32.480377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b278471-c3e1-4f7e-a87c-b0bd9093407a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:32.572925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b278471-c3e1-4f7e-a87c-b0bd9093407a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:32.605311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e583f-e1d1-46bf-8b36-c7feb4411b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:32.605820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e583f-e1d1-46bf-8b36-c7feb4411b9f None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:32.606088 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e583f-e1d1-46bf-8b36-c7feb4411b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:32.606378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e583f-e1d1-46bf-8b36-c7feb4411b9f None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:32.607021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e583f-e1d1-46bf-8b36-c7feb4411b9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:32.608412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e583f-e1d1-46bf-8b36-c7feb4411b9f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:32.629122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 419/843] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:13:32 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:13:34.798048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c4a1-c821-4b68-882d-6a3cba55960d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:34.798616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c4a1-c821-4b68-882d-6a3cba55960d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:34.798616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c4a1-c821-4b68-882d-6a3cba55960d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:34.799960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 425/844] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:34.805494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f985db5-df3d-479e-b6ab-593527cad816 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:34.805942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f985db5-df3d-479e-b6ab-593527cad816 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:34.806038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f985db5-df3d-479e-b6ab-593527cad816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:34.824137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9f985db5-df3d-479e-b6ab-593527cad816 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:34.825285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9f985db5-df3d-479e-b6ab-593527cad816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:34.844156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:34.889325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 420/845] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:35.163003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214c4a1-c821-4b68-882d-6a3cba55960d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:35.197538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:35.261891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214c4a1-c821-4b68-882d-6a3cba55960d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:35.296319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1a419-995e-420d-ad73-50faae5eee6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:35.296928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1a419-995e-420d-ad73-50faae5eee6e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:35.297200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1a419-995e-420d-ad73-50faae5eee6e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:35.297466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1a419-995e-420d-ad73-50faae5eee6e None admin] PATH_INFO: `/v3/roles/eaef1c14dd23498185dccc05b242ff6a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:35.298846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1a419-995e-420d-ad73-50faae5eee6e None admin] RBAC: Authorizing `identity:get_role(role_id=eaef1c14dd23498185dccc05b242ff6a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:35.300533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1a419-995e-420d-ad73-50faae5eee6e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:35.301903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 426/846] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:13:35 2022] GET /identity/v3/roles/eaef1c14dd23498185dccc05b242ff6a => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:35.307700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f985db5-df3d-479e-b6ab-593527cad816 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:35.403350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f985db5-df3d-479e-b6ab-593527cad816 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:35.436068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe29db-bf38-48bf-85c4-4ac2f19d869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:35.436666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe29db-bf38-48bf-85c4-4ac2f19d869f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:35.436936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe29db-bf38-48bf-85c4-4ac2f19d869f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:35.437201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe29db-bf38-48bf-85c4-4ac2f19d869f None admin] PATH_INFO: `/v3/users/ab65ec009187482c9aeabfdce2544ff0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:35.450041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe29db-bf38-48bf-85c4-4ac2f19d869f None admin] RBAC: Authorizing `identity:get_user(user_id=ab65ec009187482c9aeabfdce2544ff0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:35.451483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe29db-bf38-48bf-85c4-4ac2f19d869f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:35.453015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 421/847] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:13:35 2022] GET /identity/v3/users/ab65ec009187482c9aeabfdce2544ff0 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:35.459258 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1a419-995e-420d-ad73-50faae5eee6e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:35.555996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1a419-995e-420d-ad73-50faae5eee6e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:35.557500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:35.589578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37631cca-85a3-4e99-96a6-ae40499a4e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:35.590189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631cca-85a3-4e99-96a6-ae40499a4e39 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:35.590501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631cca-85a3-4e99-96a6-ae40499a4e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:35.590773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631cca-85a3-4e99-96a6-ae40499a4e39 None admin] PATH_INFO: `/v3/projects/bd56c4e6b5974fb1bd199b0315113281` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:35.591904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37631cca-85a3-4e99-96a6-ae40499a4e39 None admin] RBAC: Authorizing `identity:get_project(project_id=bd56c4e6b5974fb1bd199b0315113281)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:35.593322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37631cca-85a3-4e99-96a6-ae40499a4e39 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:35.594945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 427/848] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:13:35 2022] GET /identity/v3/projects/bd56c4e6b5974fb1bd199b0315113281 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:35.601877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe29db-bf38-48bf-85c4-4ac2f19d869f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:35.616933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:35.697573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe29db-bf38-48bf-85c4-4ac2f19d869f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:35.729792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a0547-a607-489e-af34-14363bffd39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:35.730440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a0547-a607-489e-af34-14363bffd39e None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:35.730710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a0547-a607-489e-af34-14363bffd39e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:35.730973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a0547-a607-489e-af34-14363bffd39e None admin] PATH_INFO: `/v3/projects/bd56c4e6b5974fb1bd199b0315113281/users/ab65ec009187482c9aeabfdce2544ff0/roles/eaef1c14dd23498185dccc05b242ff6a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:35.733017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a0547-a607-489e-af34-14363bffd39e None admin] RBAC: Authorizing `identity:create_grant(project_id=bd56c4e6b5974fb1bd199b0315113281, user_id=ab65ec009187482c9aeabfdce2544ff0, role_id=eaef1c14dd23498185dccc05b242ff6a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:35.734438 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a0547-a607-489e-af34-14363bffd39e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:35.747180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 422/849] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:13:35 2022] PUT /identity/v3/projects/bd56c4e6b5974fb1bd199b0315113281/users/ab65ec009187482c9aeabfdce2544ff0/roles/eaef1c14dd23498185dccc05b242ff6a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:13:37.864236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594c658-7a26-4d43-9b1d-d08ca2a0b1b9 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:37.864803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594c658-7a26-4d43-9b1d-d08ca2a0b1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:37.864803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594c658-7a26-4d43-9b1d-d08ca2a0b1b9 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:37.866072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 428/850] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:37.871506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12a96e-e7e9-4e7a-a431-0426b0577867 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:37.871914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12a96e-e7e9-4e7a-a431-0426b0577867 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:37.872059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12a96e-e7e9-4e7a-a431-0426b0577867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:37.890361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8c12a96e-e7e9-4e7a-a431-0426b0577867 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:37.891528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8c12a96e-e7e9-4e7a-a431-0426b0577867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:37.955228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 423/851] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:38.228662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594c658-7a26-4d43-9b1d-d08ca2a0b1b9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:38.327427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594c658-7a26-4d43-9b1d-d08ca2a0b1b9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:38.360654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec396ad7-db91-4787-978d-44da9cfa38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:38.361262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec396ad7-db91-4787-978d-44da9cfa38ee None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:38.361536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec396ad7-db91-4787-978d-44da9cfa38ee None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:38.361803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec396ad7-db91-4787-978d-44da9cfa38ee None admin] PATH_INFO: `/v3/roles/eaef1c14dd23498185dccc05b242ff6a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:38.363707 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec396ad7-db91-4787-978d-44da9cfa38ee None admin] RBAC: Authorizing `identity:get_role(role_id=eaef1c14dd23498185dccc05b242ff6a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:38.365052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec396ad7-db91-4787-978d-44da9cfa38ee None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:38.366342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 429/852] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:13:38 2022] GET /identity/v3/roles/eaef1c14dd23498185dccc05b242ff6a => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:38.371972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12a96e-e7e9-4e7a-a431-0426b0577867 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:38.375015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:38.467237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12a96e-e7e9-4e7a-a431-0426b0577867 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:38.499667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f9b78-c271-444c-a1d6-06759697ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:38.500215 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f9b78-c271-444c-a1d6-06759697ff7c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:38.500484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f9b78-c271-444c-a1d6-06759697ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:38.500744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f9b78-c271-444c-a1d6-06759697ff7c None admin] PATH_INFO: `/v3/users/ab65ec009187482c9aeabfdce2544ff0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:38.501842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f9b78-c271-444c-a1d6-06759697ff7c None admin] RBAC: Authorizing `identity:get_user(user_id=ab65ec009187482c9aeabfdce2544ff0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:38.503843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f9b78-c271-444c-a1d6-06759697ff7c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:38.505368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 424/853] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:13:38 2022] GET /identity/v3/users/ab65ec009187482c9aeabfdce2544ff0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:38.510999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec396ad7-db91-4787-978d-44da9cfa38ee None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:38.549213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:38.608585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec396ad7-db91-4787-978d-44da9cfa38ee None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:38.641648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b0f1-481d-44ca-b352-9bbc8fdabdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:38.642230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b0f1-481d-44ca-b352-9bbc8fdabdfc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:38.642530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b0f1-481d-44ca-b352-9bbc8fdabdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:38.642798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b0f1-481d-44ca-b352-9bbc8fdabdfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:38.705964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294b0f1-481d-44ca-b352-9bbc8fdabdfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:38.707529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294b0f1-481d-44ca-b352-9bbc8fdabdfc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:38.744273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:38.802146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 430/854] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:13:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:13:38.808049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f9b78-c271-444c-a1d6-06759697ff7c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:38.874428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:38.902849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f9b78-c271-444c-a1d6-06759697ff7c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:38.934544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc42bdd-91d8-4f13-961c-d41063c05f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:38.935144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc42bdd-91d8-4f13-961c-d41063c05f09 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:38.935413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc42bdd-91d8-4f13-961c-d41063c05f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:38.935673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc42bdd-91d8-4f13-961c-d41063c05f09 None admin] PATH_INFO: `/v3/projects/bd56c4e6b5974fb1bd199b0315113281` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:38.936744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc42bdd-91d8-4f13-961c-d41063c05f09 None admin] RBAC: Authorizing `identity:get_project(project_id=bd56c4e6b5974fb1bd199b0315113281)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:38.938136 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc42bdd-91d8-4f13-961c-d41063c05f09 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:38.939743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 425/855] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:13:38 2022] GET /identity/v3/projects/bd56c4e6b5974fb1bd199b0315113281 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:38.946533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b0f1-481d-44ca-b352-9bbc8fdabdfc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:39.042424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b0f1-481d-44ca-b352-9bbc8fdabdfc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:39.075408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209eb5f2-114f-409f-a41c-6bfc34f76282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:39.075950 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb5f2-114f-409f-a41c-6bfc34f76282 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:39.076226 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb5f2-114f-409f-a41c-6bfc34f76282 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:39.076494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb5f2-114f-409f-a41c-6bfc34f76282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:39.077206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209eb5f2-114f-409f-a41c-6bfc34f76282 None admin] RBAC: Adding query filter params (role.id=eaef1c14dd23498185dccc05b242ff6a, scope.project.id=bd56c4e6b5974fb1bd199b0315113281, user.id=ab65ec009187482c9aeabfdce2544ff0) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:39.077511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209eb5f2-114f-409f-a41c-6bfc34f76282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:39.079453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209eb5f2-114f-409f-a41c-6bfc34f76282 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:39.084931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 431/856] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:13:38 2022] GET /identity/v3/role_assignments?user.id=ab65ec009187482c9aeabfdce2544ff0&scope.project.id=bd56c4e6b5974fb1bd199b0315113281&role.id=eaef1c14dd23498185dccc05b242ff6a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:41.208488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04692bb-406a-4dfb-b7f2-803741de2e42 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:41.209042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04692bb-406a-4dfb-b7f2-803741de2e42 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:41.209042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04692bb-406a-4dfb-b7f2-803741de2e42 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:41.210340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 426/857] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:41.215574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ad49c-acd6-4d50-bdc3-ae246402630e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:41.215995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ad49c-acd6-4d50-bdc3-ae246402630e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:41.216113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ad49c-acd6-4d50-bdc3-ae246402630e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:41.223484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:41.236141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5c8ad49c-acd6-4d50-bdc3-ae246402630e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:41.237303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5c8ad49c-acd6-4d50-bdc3-ae246402630e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:41.301735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 432/858] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:41.574251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04692bb-406a-4dfb-b7f2-803741de2e42 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:41.673029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04692bb-406a-4dfb-b7f2-803741de2e42 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:41.704961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66102e-9a8e-40d4-b9b2-610ab0b0e73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:41.705565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66102e-9a8e-40d4-b9b2-610ab0b0e73b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:41.705839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66102e-9a8e-40d4-b9b2-610ab0b0e73b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:41.706112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66102e-9a8e-40d4-b9b2-610ab0b0e73b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:41.707253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66102e-9a8e-40d4-b9b2-610ab0b0e73b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:41.708653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66102e-9a8e-40d4-b9b2-610ab0b0e73b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:41.710181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 427/859] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:41.715984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ad49c-acd6-4d50-bdc3-ae246402630e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:41.811336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ad49c-acd6-4d50-bdc3-ae246402630e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:41.839412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:41.844533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aa55a-d39e-4101-83cb-b5be3de0a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:41.845124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aa55a-d39e-4101-83cb-b5be3de0a4b5 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:41.845401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aa55a-d39e-4101-83cb-b5be3de0a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:41.845673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aa55a-d39e-4101-83cb-b5be3de0a4b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:41.846319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aa55a-d39e-4101-83cb-b5be3de0a4b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:41.848683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aa55a-d39e-4101-83cb-b5be3de0a4b5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:41.875512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 433/860] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:13:41 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:13:44.023833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33060-da75-44ef-b962-13d675834abb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:44.024429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33060-da75-44ef-b962-13d675834abb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:44.024429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33060-da75-44ef-b962-13d675834abb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:44.025679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 428/861] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:44.030848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c43ba-844e-4f39-8412-c4989e1819e8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:44.031282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c43ba-844e-4f39-8412-c4989e1819e8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:44.031391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c43ba-844e-4f39-8412-c4989e1819e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:44.049598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2d2c43ba-844e-4f39-8412-c4989e1819e8 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:44.051498 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2d2c43ba-844e-4f39-8412-c4989e1819e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:44.116893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 434/862] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:44.385883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf33060-da75-44ef-b962-13d675834abb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:44.397032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:44.485418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf33060-da75-44ef-b962-13d675834abb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:44.518046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d235c58-7320-4651-8d2c-3374e90efc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:44.519212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d235c58-7320-4651-8d2c-3374e90efc10 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:44.519740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d235c58-7320-4651-8d2c-3374e90efc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:44.519740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d235c58-7320-4651-8d2c-3374e90efc10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:44.520817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d235c58-7320-4651-8d2c-3374e90efc10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:44.522218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d235c58-7320-4651-8d2c-3374e90efc10 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:44.523774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 429/863] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:13:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:44.529479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c43ba-844e-4f39-8412-c4989e1819e8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:44.624753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c43ba-844e-4f39-8412-c4989e1819e8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:44.638587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:44.659928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef8c08-26f8-4650-8304-93cea5333da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:44.660510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef8c08-26f8-4650-8304-93cea5333da6 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:44.660779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef8c08-26f8-4650-8304-93cea5333da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:44.661042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef8c08-26f8-4650-8304-93cea5333da6 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:44.661686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef8c08-26f8-4650-8304-93cea5333da6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:44.663205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef8c08-26f8-4650-8304-93cea5333da6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:44.683307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 435/864] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:13:44 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:13:46.830548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8b5f7-b2c1-42d6-bf85-7dd4ebf6040f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:46.831144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8b5f7-b2c1-42d6-bf85-7dd4ebf6040f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:46.831144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8b5f7-b2c1-42d6-bf85-7dd4ebf6040f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:46.832445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 430/865] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:46.837597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000db5f-fa6b-41e1-9bf0-cd9917c24a7c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:46.838027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000db5f-fa6b-41e1-9bf0-cd9917c24a7c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:46.838120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000db5f-fa6b-41e1-9bf0-cd9917c24a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:46.856592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1000db5f-fa6b-41e1-9bf0-cd9917c24a7c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:46.857733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1000db5f-fa6b-41e1-9bf0-cd9917c24a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:46.923181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 436/866] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:47.189915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8b5f7-b2c1-42d6-bf85-7dd4ebf6040f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:47.264966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:47.289396 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8b5f7-b2c1-42d6-bf85-7dd4ebf6040f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:47.321071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bf9c-d3ee-4e32-abbb-3aba61c93303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:47.321679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bf9c-d3ee-4e32-abbb-3aba61c93303 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:47.321951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bf9c-d3ee-4e32-abbb-3aba61c93303 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:47.322218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bf9c-d3ee-4e32-abbb-3aba61c93303 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:47.332479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7bf9c-d3ee-4e32-abbb-3aba61c93303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d2a7bf9c-d3ee-4e32-abbb-3aba61c93303 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:13:47.337200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:47.338288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:13:47.339517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 431/867] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:47.344177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000db5f-fa6b-41e1-9bf0-cd9917c24a7c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:47.437612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:47.440985 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000db5f-fa6b-41e1-9bf0-cd9917c24a7c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:47.477014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:47.477626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:47.477926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:47.478232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:47.478966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:47.479271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:47.480748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:47.488976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 437/868] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:47.494776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bf9c-d3ee-4e32-abbb-3aba61c93303 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:47.596422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bf9c-d3ee-4e32-abbb-3aba61c93303 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:47.630124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d71ee5-49b3-43f2-bdd6-0d89d6ec7d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:47.630760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d71ee5-49b3-43f2-bdd6-0d89d6ec7d46 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:47.631055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d71ee5-49b3-43f2-bdd6-0d89d6ec7d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:47.631302 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d71ee5-49b3-43f2-bdd6-0d89d6ec7d46 None admin] PATH_INFO: `/v3/users/5ca51403d65f4427a21ae3a7c33cb6ba` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:47.633839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:47.644804 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d71ee5-49b3-43f2-bdd6-0d89d6ec7d46 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca51403d65f4427a21ae3a7c33cb6ba)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:47.646221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d71ee5-49b3-43f2-bdd6-0d89d6ec7d46 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:47.647827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 432/869] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:13:47 2022] GET /identity/v3/users/5ca51403d65f4427a21ae3a7c33cb6ba => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:47.653792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:47.749598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239de1e-6c37-4cc6-9236-0952085975bd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:47.782473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12451f5c-6f3f-4588-b494-c04bb553d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:47.783081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12451f5c-6f3f-4588-b494-c04bb553d31c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:47.783353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12451f5c-6f3f-4588-b494-c04bb553d31c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:47.783620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12451f5c-6f3f-4588-b494-c04bb553d31c None admin] PATH_INFO: `/v3/projects/57edd9c7e8cd4fe382aeca7cc68455fd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:47.784915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12451f5c-6f3f-4588-b494-c04bb553d31c None admin] RBAC: Authorizing `identity:get_project(project_id=57edd9c7e8cd4fe382aeca7cc68455fd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:47.786311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12451f5c-6f3f-4588-b494-c04bb553d31c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:47.787921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 438/870] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:13:47 2022] GET /identity/v3/projects/57edd9c7e8cd4fe382aeca7cc68455fd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:47.793780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d71ee5-49b3-43f2-bdd6-0d89d6ec7d46 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:47.890800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d71ee5-49b3-43f2-bdd6-0d89d6ec7d46 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:47.923334 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069deb3-8d53-4b65-828c-9e198dd4a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:47.923965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069deb3-8d53-4b65-828c-9e198dd4a1cf None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:47.924240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069deb3-8d53-4b65-828c-9e198dd4a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:47.924509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069deb3-8d53-4b65-828c-9e198dd4a1cf None admin] PATH_INFO: `/v3/projects/57edd9c7e8cd4fe382aeca7cc68455fd/users/5ca51403d65f4427a21ae3a7c33cb6ba/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:47.926632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069deb3-8d53-4b65-828c-9e198dd4a1cf None admin] RBAC: Authorizing `identity:create_grant(project_id=57edd9c7e8cd4fe382aeca7cc68455fd, user_id=5ca51403d65f4427a21ae3a7c33cb6ba, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:47.928041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069deb3-8d53-4b65-828c-9e198dd4a1cf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:47.940045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 433/871] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:13:47 2022] PUT /identity/v3/projects/57edd9c7e8cd4fe382aeca7cc68455fd/users/5ca51403d65f4427a21ae3a7c33cb6ba/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:13:50.083027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520747-eb6e-4c32-a08a-1b8c1854649c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:50.083471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520747-eb6e-4c32-a08a-1b8c1854649c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:50.083561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520747-eb6e-4c32-a08a-1b8c1854649c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:50.084897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 439/872] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:50.090242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63935-9078-4c90-a552-55ddf57ca2cf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:50.090675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63935-9078-4c90-a552-55ddf57ca2cf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:50.090807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63935-9078-4c90-a552-55ddf57ca2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:50.108913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f8b63935-9078-4c90-a552-55ddf57ca2cf None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:50.110060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f8b63935-9078-4c90-a552-55ddf57ca2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:50.158236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:50.176256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 434/873] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:50.454972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20520747-eb6e-4c32-a08a-1b8c1854649c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:50.485731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:50.553973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20520747-eb6e-4c32-a08a-1b8c1854649c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:50.586281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63184497-13cd-4258-b9c0-831ba08ffd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:50.586913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63184497-13cd-4258-b9c0-831ba08ffd49 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:50.587186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63184497-13cd-4258-b9c0-831ba08ffd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:50.587448 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63184497-13cd-4258-b9c0-831ba08ffd49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:50.597685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63184497-13cd-4258-b9c0-831ba08ffd49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-63184497-13cd-4258-b9c0-831ba08ffd49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:13:50.602638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:13:50.603703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:13:50.604583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 440/874] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:13:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:13:50.609632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63935-9078-4c90-a552-55ddf57ca2cf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:50.703061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63935-9078-4c90-a552-55ddf57ca2cf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:50.735647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:50.736170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:50.736443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:50.736710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:50.737414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:50.737800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:50.739112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:50.746833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 435/875] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:13:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:50.752549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63184497-13cd-4258-b9c0-831ba08ffd49 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:50.840214 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:50.849932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63184497-13cd-4258-b9c0-831ba08ffd49 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:50.882107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a04aa-3dd1-46a6-9ecd-90ff011f637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:50.882729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a04aa-3dd1-46a6-9ecd-90ff011f637f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:50.882997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a04aa-3dd1-46a6-9ecd-90ff011f637f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:50.883259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a04aa-3dd1-46a6-9ecd-90ff011f637f None admin] PATH_INFO: `/v3/users/5ca51403d65f4427a21ae3a7c33cb6ba` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:50.884339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a04aa-3dd1-46a6-9ecd-90ff011f637f None admin] RBAC: Authorizing `identity:get_user(user_id=5ca51403d65f4427a21ae3a7c33cb6ba)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:50.885747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a04aa-3dd1-46a6-9ecd-90ff011f637f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:50.887287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 441/876] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:13:50 2022] GET /identity/v3/users/5ca51403d65f4427a21ae3a7c33cb6ba => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:50.893093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:50.938483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:50.990923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fe57b-e91d-4193-bb06-08cc63ff8904 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:51.023231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d08982-b32c-4742-99b2-657e72e6cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:51.023812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d08982-b32c-4742-99b2-657e72e6cc7e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:51.024087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d08982-b32c-4742-99b2-657e72e6cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:51.024354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d08982-b32c-4742-99b2-657e72e6cc7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:51.090858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d08982-b32c-4742-99b2-657e72e6cc7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:51.092267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d08982-b32c-4742-99b2-657e72e6cc7e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:51.135309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:51.186602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 436/877] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:13:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:13:51.193668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a04aa-3dd1-46a6-9ecd-90ff011f637f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:51.287657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a04aa-3dd1-46a6-9ecd-90ff011f637f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:51.320075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8515469-44f6-41c4-b83d-d76515ed3375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:51.320676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8515469-44f6-41c4-b83d-d76515ed3375 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:51.320948 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8515469-44f6-41c4-b83d-d76515ed3375 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:51.321212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8515469-44f6-41c4-b83d-d76515ed3375 None admin] PATH_INFO: `/v3/projects/57edd9c7e8cd4fe382aeca7cc68455fd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:51.322287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8515469-44f6-41c4-b83d-d76515ed3375 None admin] RBAC: Authorizing `identity:get_project(project_id=57edd9c7e8cd4fe382aeca7cc68455fd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:51.323725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8515469-44f6-41c4-b83d-d76515ed3375 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:51.325285 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 442/878] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:13:51 2022] GET /identity/v3/projects/57edd9c7e8cd4fe382aeca7cc68455fd => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:51.331633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d08982-b32c-4742-99b2-657e72e6cc7e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:51.428973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d08982-b32c-4742-99b2-657e72e6cc7e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:51.460589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e7d19-1182-44f6-aa77-cbc9725df598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:51.461156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e7d19-1182-44f6-aa77-cbc9725df598 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:51.461424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e7d19-1182-44f6-aa77-cbc9725df598 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:51.461688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e7d19-1182-44f6-aa77-cbc9725df598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:51.462419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e7d19-1182-44f6-aa77-cbc9725df598 None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=57edd9c7e8cd4fe382aeca7cc68455fd, user.id=5ca51403d65f4427a21ae3a7c33cb6ba) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:13:51.462719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e7d19-1182-44f6-aa77-cbc9725df598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:51.464105 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e7d19-1182-44f6-aa77-cbc9725df598 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:51.469560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 437/879] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:13:51 2022] GET /identity/v3/role_assignments?user.id=5ca51403d65f4427a21ae3a7c33cb6ba&scope.project.id=57edd9c7e8cd4fe382aeca7cc68455fd&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:53.643572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6ee1-41ec-4379-8e11-019491747ff5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:53.644709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6ee1-41ec-4379-8e11-019491747ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:53.644709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6ee1-41ec-4379-8e11-019491747ff5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:53.645907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 443/880] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:53.651086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf94776-8c67-4766-8771-b980bc4e00bd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:53.651519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf94776-8c67-4766-8771-b980bc4e00bd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:53.651620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf94776-8c67-4766-8771-b980bc4e00bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:53.669356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0cf94776-8c67-4766-8771-b980bc4e00bd None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:53.670551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0cf94776-8c67-4766-8771-b980bc4e00bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:53.673220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:53.736627 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 438/881] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:54.007445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f6ee1-41ec-4379-8e11-019491747ff5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:54.035945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:54.111855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f6ee1-41ec-4379-8e11-019491747ff5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:54.144759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9feef4-c390-428d-9338-58d6aea3654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:54.145356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9feef4-c390-428d-9338-58d6aea3654a None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:54.145629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9feef4-c390-428d-9338-58d6aea3654a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:54.145897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9feef4-c390-428d-9338-58d6aea3654a None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:54.146591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9feef4-c390-428d-9338-58d6aea3654a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:54.147947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9feef4-c390-428d-9338-58d6aea3654a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:54.163488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 444/882] 10.222.0.16 () {60 vars in 1138 bytes} [Mon Oct 17 10:13:54 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:56.302119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d37e7-1131-4152-9bff-949e5d88b567 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:56.302723 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d37e7-1131-4152-9bff-949e5d88b567 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:56.302723 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d37e7-1131-4152-9bff-949e5d88b567 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:56.304012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 439/883] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:56.309160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ea9c-b88b-490d-a596-c1ec92644278 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:56.309550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ea9c-b88b-490d-a596-c1ec92644278 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:56.309697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ea9c-b88b-490d-a596-c1ec92644278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:56.327813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4635ea9c-b88b-490d-a596-c1ec92644278 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:56.329016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4635ea9c-b88b-490d-a596-c1ec92644278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:56.392220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 445/884] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:56.656664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d37e7-1131-4152-9bff-949e5d88b567 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:56.755514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d37e7-1131-4152-9bff-949e5d88b567 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:56.787881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79ffa-1343-4526-9d24-cd32ed1085d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:56.788490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79ffa-1343-4526-9d24-cd32ed1085d1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:56.788757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79ffa-1343-4526-9d24-cd32ed1085d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:56.789025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79ffa-1343-4526-9d24-cd32ed1085d1 None admin] PATH_INFO: `/v3/domains/6344184d4d2f4f2990a67f09f119f59c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:56.797012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:56.799849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d79ffa-1343-4526-9d24-cd32ed1085d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6344184d4d2f4f2990a67f09f119f59c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:56.801561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d79ffa-1343-4526-9d24-cd32ed1085d1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:56.803361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 440/885] 10.222.0.16 () {56 vars in 1183 bytes} [Mon Oct 17 10:13:56 2022] GET /identity/v3/domains/6344184d4d2f4f2990a67f09f119f59c => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:56.809121 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635ea9c-b88b-490d-a596-c1ec92644278 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:56.826703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:56.906966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635ea9c-b88b-490d-a596-c1ec92644278 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:56.938697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ada6f4-92ee-4068-a7dc-2fa5e565ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:56.939286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ada6f4-92ee-4068-a7dc-2fa5e565ffa5 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:56.939560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ada6f4-92ee-4068-a7dc-2fa5e565ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:56.939826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ada6f4-92ee-4068-a7dc-2fa5e565ffa5 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:56.941050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ada6f4-92ee-4068-a7dc-2fa5e565ffa5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:56.942492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ada6f4-92ee-4068-a7dc-2fa5e565ffa5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:56.969056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 446/886] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:13:56 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:13:59.104195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b0e56-ac50-40ab-87a9-04de1f1f1621 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:59.104686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b0e56-ac50-40ab-87a9-04de1f1f1621 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:59.104794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b0e56-ac50-40ab-87a9-04de1f1f1621 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:59.106057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 441/887] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:13:59.111183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187c9dc-f1da-4621-9b16-9dd00c10fde8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:59.111585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187c9dc-f1da-4621-9b16-9dd00c10fde8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:59.111716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187c9dc-f1da-4621-9b16-9dd00c10fde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:59.130390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9187c9dc-f1da-4621-9b16-9dd00c10fde8 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:13:59.131530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9187c9dc-f1da-4621-9b16-9dd00c10fde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:13:59.189282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:13:59.196131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 447/888] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:13:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:13:59.461710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b0e56-ac50-40ab-87a9-04de1f1f1621 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:59.561099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b0e56-ac50-40ab-87a9-04de1f1f1621 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:59.592736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79673b3-c553-47ee-9b90-07f0d0797635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:59.593336 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79673b3-c553-47ee-9b90-07f0d0797635 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:59.593607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79673b3-c553-47ee-9b90-07f0d0797635 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:59.593873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79673b3-c553-47ee-9b90-07f0d0797635 None admin] PATH_INFO: `/v3/domains/6344184d4d2f4f2990a67f09f119f59c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:59.594978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79673b3-c553-47ee-9b90-07f0d0797635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6344184d4d2f4f2990a67f09f119f59c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:59.596404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79673b3-c553-47ee-9b90-07f0d0797635 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:59.598520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 442/889] 10.222.0.16 () {56 vars in 1183 bytes} [Mon Oct 17 10:13:59 2022] GET /identity/v3/domains/6344184d4d2f4f2990a67f09f119f59c => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:13:59.604114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187c9dc-f1da-4621-9b16-9dd00c10fde8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:13:59.700875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187c9dc-f1da-4621-9b16-9dd00c10fde8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:13:59.734677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f4e4f-3c82-4cd8-8fad-18907804d69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:13:59.735183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4e4f-3c82-4cd8-8fad-18907804d69b None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:13:59.735456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4e4f-3c82-4cd8-8fad-18907804d69b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:13:59.735725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4e4f-3c82-4cd8-8fad-18907804d69b None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:13:59.736363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f4e4f-3c82-4cd8-8fad-18907804d69b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:13:59.738569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f4e4f-3c82-4cd8-8fad-18907804d69b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:13:59.759632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 448/890] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:13:59 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:14:01.925141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373e525-4e70-4e97-97b3-7c38aa3f2d88 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:01.925718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373e525-4e70-4e97-97b3-7c38aa3f2d88 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:01.925718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373e525-4e70-4e97-97b3-7c38aa3f2d88 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:01.927020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 443/891] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:14:01.932109 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e99f-f1c9-496c-9b79-bb1cc611c15b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:01.932547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e99f-f1c9-496c-9b79-bb1cc611c15b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:01.932642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e99f-f1c9-496c-9b79-bb1cc611c15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:01.950697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2b36e99f-f1c9-496c-9b79-bb1cc611c15b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:14:01.951854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2b36e99f-f1c9-496c-9b79-bb1cc611c15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:14:01.999050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:02.017482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 449/892] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:14:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:14:02.293310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373e525-4e70-4e97-97b3-7c38aa3f2d88 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:02.358764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:02.392295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373e525-4e70-4e97-97b3-7c38aa3f2d88 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:02.423206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f0421-710b-4bb5-ab0a-5fc1965d28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:02.423800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f0421-710b-4bb5-ab0a-5fc1965d28ef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:02.424068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f0421-710b-4bb5-ab0a-5fc1965d28ef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:02.424331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f0421-710b-4bb5-ab0a-5fc1965d28ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:02.434970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f0421-710b-4bb5-ab0a-5fc1965d28ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7d6f0421-710b-4bb5-ab0a-5fc1965d28ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:14:02.439585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:14:02.440803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:14:02.442954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 444/893] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:14:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:14:02.447283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36e99f-f1c9-496c-9b79-bb1cc611c15b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:02.540956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36e99f-f1c9-496c-9b79-bb1cc611c15b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:02.572644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:02.573166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:02.573437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:02.573703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:02.574366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:14:02.574665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:02.576003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:02.584165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 450/894] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:14:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:14:02.589903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f0421-710b-4bb5-ab0a-5fc1965d28ef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:02.684999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f0421-710b-4bb5-ab0a-5fc1965d28ef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:02.710539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:02.718272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d415bf-e21f-45f8-ad7f-2f3a904b4b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:02.718896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d415bf-e21f-45f8-ad7f-2f3a904b4b59 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:02.719167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d415bf-e21f-45f8-ad7f-2f3a904b4b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:02.719432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d415bf-e21f-45f8-ad7f-2f3a904b4b59 None admin] PATH_INFO: `/v3/users/4d97634c6cb54a6ca5ca2b8f56f1a337` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:02.732768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d415bf-e21f-45f8-ad7f-2f3a904b4b59 None admin] RBAC: Authorizing `identity:get_user(user_id=4d97634c6cb54a6ca5ca2b8f56f1a337)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:02.734173 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d415bf-e21f-45f8-ad7f-2f3a904b4b59 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:02.735756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 445/895] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:14:02 2022] GET /identity/v3/users/4d97634c6cb54a6ca5ca2b8f56f1a337 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:14:02.741756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:02.786983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:02.838573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77aa6-8434-4fb1-a28f-4be173873ea9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:02.870081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f4c04-f74f-4ab2-9978-97be0aae08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:02.870711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4c04-f74f-4ab2-9978-97be0aae08d2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:02.870988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4c04-f74f-4ab2-9978-97be0aae08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:02.871255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4c04-f74f-4ab2-9978-97be0aae08d2 None admin] PATH_INFO: `/v3/projects/9b5e70bce5b8475f99a96c5d16482fe2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:02.872335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f4c04-f74f-4ab2-9978-97be0aae08d2 None admin] RBAC: Authorizing `identity:get_project(project_id=9b5e70bce5b8475f99a96c5d16482fe2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:02.873741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f4c04-f74f-4ab2-9978-97be0aae08d2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:02.875348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 451/896] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:14:02 2022] GET /identity/v3/projects/9b5e70bce5b8475f99a96c5d16482fe2 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:14:02.881996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d415bf-e21f-45f8-ad7f-2f3a904b4b59 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:02.976073 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d415bf-e21f-45f8-ad7f-2f3a904b4b59 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:03.007730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2aab07-42df-4c97-a269-3cd908825b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:03.008362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aab07-42df-4c97-a269-3cd908825b0c None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:03.008634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aab07-42df-4c97-a269-3cd908825b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:03.008900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aab07-42df-4c97-a269-3cd908825b0c None admin] PATH_INFO: `/v3/projects/9b5e70bce5b8475f99a96c5d16482fe2/users/4d97634c6cb54a6ca5ca2b8f56f1a337/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:03.010750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2aab07-42df-4c97-a269-3cd908825b0c None admin] RBAC: Authorizing `identity:create_grant(project_id=9b5e70bce5b8475f99a96c5d16482fe2, user_id=4d97634c6cb54a6ca5ca2b8f56f1a337, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:03.012140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2aab07-42df-4c97-a269-3cd908825b0c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:03.024150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 446/897] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:14:02 2022] PUT /identity/v3/projects/9b5e70bce5b8475f99a96c5d16482fe2/users/4d97634c6cb54a6ca5ca2b8f56f1a337/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:14:05.120295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee587-715e-4ddb-80f7-d6ebb4c1b092 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:05.120774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee587-715e-4ddb-80f7-d6ebb4c1b092 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:05.120864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee587-715e-4ddb-80f7-d6ebb4c1b092 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:05.122177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 452/898] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:14:05.127263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49ba9-c59f-4c28-b272-d347c6fa119e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:05.127711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49ba9-c59f-4c28-b272-d347c6fa119e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:05.127895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49ba9-c59f-4c28-b272-d347c6fa119e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:05.145347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-94e49ba9-c59f-4c28-b272-d347c6fa119e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:14:05.146535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94e49ba9-c59f-4c28-b272-d347c6fa119e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:14:05.186518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:05.210380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 447/899] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:14:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:14:05.478248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ee587-715e-4ddb-80f7-d6ebb4c1b092 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:05.576963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ee587-715e-4ddb-80f7-d6ebb4c1b092 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:05.590941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:05.610376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f069f34-cb7b-4874-ad74-2f8cefc4fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:05.610988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f069f34-cb7b-4874-ad74-2f8cefc4fe5d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:05.611265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f069f34-cb7b-4874-ad74-2f8cefc4fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:05.611530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f069f34-cb7b-4874-ad74-2f8cefc4fe5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:05.621871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f069f34-cb7b-4874-ad74-2f8cefc4fe5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5f069f34-cb7b-4874-ad74-2f8cefc4fe5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:14:05.626644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:14:05.628335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:14:05.629242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 453/900] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:14:05.634421 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49ba9-c59f-4c28-b272-d347c6fa119e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:05.727866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49ba9-c59f-4c28-b272-d347c6fa119e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:05.761055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:05.761646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:05.761916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:05.762179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:05.762867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:14:05.763160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:05.764504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:05.772696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 448/901] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:14:05.778696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f069f34-cb7b-4874-ad74-2f8cefc4fe5d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:05.882092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f069f34-cb7b-4874-ad74-2f8cefc4fe5d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:05.916108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6af3d-5dd9-4890-a0e1-5f9c001edab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:05.916732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6af3d-5dd9-4890-a0e1-5f9c001edab8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:05.917008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6af3d-5dd9-4890-a0e1-5f9c001edab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:05.917277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6af3d-5dd9-4890-a0e1-5f9c001edab8 None admin] PATH_INFO: `/v3/users/4d97634c6cb54a6ca5ca2b8f56f1a337` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:05.918420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6af3d-5dd9-4890-a0e1-5f9c001edab8 None admin] RBAC: Authorizing `identity:get_user(user_id=4d97634c6cb54a6ca5ca2b8f56f1a337)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:05.919867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6af3d-5dd9-4890-a0e1-5f9c001edab8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:05.921431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 454/902] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Oct 17 10:14:05 2022] GET /identity/v3/users/4d97634c6cb54a6ca5ca2b8f56f1a337 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:14:05.927934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:05.967272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:06.026618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e22618-dc7e-4999-bbd7-8f0ebafea038 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:06.058654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e179065-f1ca-4333-9364-96b6749e1193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:06.059241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e179065-f1ca-4333-9364-96b6749e1193 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:06.059513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e179065-f1ca-4333-9364-96b6749e1193 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:06.059781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e179065-f1ca-4333-9364-96b6749e1193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:06.124368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e179065-f1ca-4333-9364-96b6749e1193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:06.125791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e179065-f1ca-4333-9364-96b6749e1193 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:06.162024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:06.220256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 449/903] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:14:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:14:06.226655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6af3d-5dd9-4890-a0e1-5f9c001edab8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:06.268596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:06.322913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6af3d-5dd9-4890-a0e1-5f9c001edab8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:06.355002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190d4c7-c2ef-4768-883c-8c5982443414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:06.355607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190d4c7-c2ef-4768-883c-8c5982443414 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:06.355882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190d4c7-c2ef-4768-883c-8c5982443414 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:06.356149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190d4c7-c2ef-4768-883c-8c5982443414 None admin] PATH_INFO: `/v3/projects/9b5e70bce5b8475f99a96c5d16482fe2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:06.357241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190d4c7-c2ef-4768-883c-8c5982443414 None admin] RBAC: Authorizing `identity:get_project(project_id=9b5e70bce5b8475f99a96c5d16482fe2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:06.358686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190d4c7-c2ef-4768-883c-8c5982443414 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:06.360261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 455/904] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Oct 17 10:14:06 2022] GET /identity/v3/projects/9b5e70bce5b8475f99a96c5d16482fe2 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:14:06.366948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e179065-f1ca-4333-9364-96b6749e1193 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:14:06.461547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e179065-f1ca-4333-9364-96b6749e1193 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:14:06.494786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eaa04f-2646-4229-8908-0735c5d3e1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:14:06.495371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaa04f-2646-4229-8908-0735c5d3e1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:14:06.495648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaa04f-2646-4229-8908-0735c5d3e1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:14:06.495918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaa04f-2646-4229-8908-0735c5d3e1b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:14:06.496629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eaa04f-2646-4229-8908-0735c5d3e1b7 None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=9b5e70bce5b8475f99a96c5d16482fe2, user.id=4d97634c6cb54a6ca5ca2b8f56f1a337) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:14:06.496932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eaa04f-2646-4229-8908-0735c5d3e1b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:14:06.498352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eaa04f-2646-4229-8908-0735c5d3e1b7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:14:06.503990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 450/905] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:14:06 2022] GET /identity/v3/role_assignments?user.id=4d97634c6cb54a6ca5ca2b8f56f1a337&scope.project.id=9b5e70bce5b8475f99a96c5d16482fe2&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:14:16.351098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=45 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:14:16.501295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:02.152934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaf411-d52e-481a-8473-6fc1e651ed09 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:02.153542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaf411-d52e-481a-8473-6fc1e651ed09 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:02.153542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaf411-d52e-481a-8473-6fc1e651ed09 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:02.155172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 456/906] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:15:02.161470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482897d0-1985-448d-8a4c-e3edad4f8694 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:02.162055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482897d0-1985-448d-8a4c-e3edad4f8694 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:02.162055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482897d0-1985-448d-8a4c-e3edad4f8694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:02.184307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-482897d0-1985-448d-8a4c-e3edad4f8694 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:02.185597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-482897d0-1985-448d-8a4c-e3edad4f8694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:02.288467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 451/907] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:15:02.562294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddaf411-d52e-481a-8473-6fc1e651ed09 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:02.691014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddaf411-d52e-481a-8473-6fc1e651ed09 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:02.741829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:02.742601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:02.742905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:02.743216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:02.755984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:02.757637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:02.774838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:02.776166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:02.777592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:02.777592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:02.777592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:02.784557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 457/908] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:02.784741 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482897d0-1985-448d-8a4c-e3edad4f8694 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:02.901953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482897d0-1985-448d-8a4c-e3edad4f8694 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:02.937733 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:02.947053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589ffc9-632f-4258-b16c-b5106af26c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:02.947702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ffc9-632f-4258-b16c-b5106af26c6d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:02.947974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ffc9-632f-4258-b16c-b5106af26c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:02.948332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ffc9-632f-4258-b16c-b5106af26c6d None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:02.948985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589ffc9-632f-4258-b16c-b5106af26c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:02.949315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589ffc9-632f-4258-b16c-b5106af26c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:02.950737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589ffc9-632f-4258-b16c-b5106af26c6d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:02.977409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 452/909] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:15:02.979170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:03.053418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:03.124897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5759d-9759-46e4-9edb-e1af6cf055ba None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:03.181536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54815f-a71c-43a2-b4a6-482138576221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:03.182198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54815f-a71c-43a2-b4a6-482138576221 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:03.182840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54815f-a71c-43a2-b4a6-482138576221 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:03.182840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54815f-a71c-43a2-b4a6-482138576221 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:03.184160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54815f-a71c-43a2-b4a6-482138576221 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:03.186091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54815f-a71c-43a2-b4a6-482138576221 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:03.221051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 458/910] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Oct 17 10:15:02 2022] POST /identity/v3/users => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:15:05.802297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6eb18-5820-4148-83c9-3653dd4e0b36 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:05.802906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6eb18-5820-4148-83c9-3653dd4e0b36 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:05.802906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6eb18-5820-4148-83c9-3653dd4e0b36 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:05.804177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 453/911] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:15:05.809836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374f528-4086-4c02-8436-548ed930a0a9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:05.810348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374f528-4086-4c02-8436-548ed930a0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:05.810444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374f528-4086-4c02-8436-548ed930a0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:05.832681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1374f528-4086-4c02-8436-548ed930a0a9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:05.834086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1374f528-4086-4c02-8436-548ed930a0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:05.925107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 459/912] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:15:06.244335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6eb18-5820-4148-83c9-3653dd4e0b36 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:06.397258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6eb18-5820-4148-83c9-3653dd4e0b36 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:06.441691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9e2f6-76b4-4726-a9fd-80dd80295eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:06.442571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9e2f6-76b4-4726-a9fd-80dd80295eca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:06.443463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9e2f6-76b4-4726-a9fd-80dd80295eca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:06.443463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9e2f6-76b4-4726-a9fd-80dd80295eca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:06.455878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9e2f6-76b4-4726-a9fd-80dd80295eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8fd9e2f6-76b4-4726-a9fd-80dd80295eca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:15:06.460682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:06.462292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:06.463673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 454/913] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:15:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:06.467822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374f528-4086-4c02-8436-548ed930a0a9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:06.517335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:06.585981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374f528-4086-4c02-8436-548ed930a0a9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:06.627802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:06.628449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:06.628727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:06.628994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:06.629666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:06.629962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:06.631485 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:06.640199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 460/914] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:15:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:06.646253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9e2f6-76b4-4726-a9fd-80dd80295eca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:06.704056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:06.770529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9e2f6-76b4-4726-a9fd-80dd80295eca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:06.812868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:06.814440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:06.814440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:06.814666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:06.827331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:06.828788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:06.846160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:06.847440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:06.848844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:06.848844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:06.848844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:06.848844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 455/915] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:06.853281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:06.976903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:06.979930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab6134-eff3-4ead-a78a-5783471bd740 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:07.019015 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:07.019693 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:07.019974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:07.020238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:07.020890 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:07.021189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:07.022612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:07.040988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 461/916] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:15:07.042240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:07.152628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8b8b-5333-43ee-a02e-291ef4c9d84c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:07.154167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:07.193087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:07.193910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:07.194253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:07.194611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:07.205099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:07.207202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:15:07.220451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:07.222023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:07.223151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 456/917] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:07.228344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:07.339238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646c7d7-90d7-448a-83ad-754d4e27a44a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:07.379343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:07.380061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:07.380315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:07.380573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:07.381383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:07.381743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:07.383217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:07.406775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 462/918] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Oct 17 10:15:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:07.408948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:07.518834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341affd-b90d-41d4-b4e3-873c0f094817 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:07.556777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:07.557477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:07.557878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:07.558209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:07.570689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:07.572208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:07.586648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:07.587812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:07.588839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:07.588839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:07.588839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:07.588839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 457/919] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:07.594611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:07.617694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:07.713429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7c137-b993-44e5-8b95-5180b5ba29b0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:07.753223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:07.754139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:07.754507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:07.754800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:07.755605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:07.755943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:07.757598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:07.779146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 463/920] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:07.781964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:07.801583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:07.914490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e7803-77d3-4de9-ab27-6b6d7aa7e788 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:07.952040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:07.952757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:07.953029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:07.953415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:07.962774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:07.964284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:15:07.976603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:07.984359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:07.985554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 458/921] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:07.986186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:08.089349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:08.116475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9e97b-9351-4991-938e-168d09414a15 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:08.161256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58446c-8af8-472c-addf-4eb8e87c6074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:08.162061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58446c-8af8-472c-addf-4eb8e87c6074 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:08.162343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58446c-8af8-472c-addf-4eb8e87c6074 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:08.162623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58446c-8af8-472c-addf-4eb8e87c6074 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:08.163428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58446c-8af8-472c-addf-4eb8e87c6074 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:08.163759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58446c-8af8-472c-addf-4eb8e87c6074 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:08.165423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58446c-8af8-472c-addf-4eb8e87c6074 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:08.179649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 464/922] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:15:08.190386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:08.283461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:08.315901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f2fa5-7c21-4dee-859d-fd50a7e7782c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:08.354934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388710b7-83af-4e82-98d4-9b5d8d16df27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:08.355493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388710b7-83af-4e82-98d4-9b5d8d16df27 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:08.355765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388710b7-83af-4e82-98d4-9b5d8d16df27 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:08.356039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388710b7-83af-4e82-98d4-9b5d8d16df27 None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/dbb92e22612b400dbf390e149abbb105/roles/4045fa40bdd54e9f889b47e1fb2704f5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:08.372578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388710b7-83af-4e82-98d4-9b5d8d16df27 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=dbb92e22612b400dbf390e149abbb105, role_id=4045fa40bdd54e9f889b47e1fb2704f5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:08.374156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388710b7-83af-4e82-98d4-9b5d8d16df27 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:08.387858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 459/923] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:15:08 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/dbb92e22612b400dbf390e149abbb105/roles/4045fa40bdd54e9f889b47e1fb2704f5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:15:10.663834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ad3c0-2ce6-4a24-963a-fd7bb3f9721b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:10.664585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ad3c0-2ce6-4a24-963a-fd7bb3f9721b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:10.664585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ad3c0-2ce6-4a24-963a-fd7bb3f9721b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:10.665874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 465/924] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:15:10.672074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9179207-0f38-4662-900d-764a1507456e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:10.672539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9179207-0f38-4662-900d-764a1507456e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:10.672632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9179207-0f38-4662-900d-764a1507456e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:10.692697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b9179207-0f38-4662-900d-764a1507456e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:10.693971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b9179207-0f38-4662-900d-764a1507456e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:10.771803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 460/925] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:15:11.056270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ad3c0-2ce6-4a24-963a-fd7bb3f9721b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:11.172184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ad3c0-2ce6-4a24-963a-fd7bb3f9721b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:11.211384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5c3a-af37-4d61-9b48-1e34c33021fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:11.212020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5c3a-af37-4d61-9b48-1e34c33021fd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:11.212300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5c3a-af37-4d61-9b48-1e34c33021fd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:11.212572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5c3a-af37-4d61-9b48-1e34c33021fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:11.224576 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:11.229325 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef5c3a-af37-4d61-9b48-1e34c33021fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9bef5c3a-af37-4d61-9b48-1e34c33021fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:15:11.236855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:11.238827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:11.240287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 466/926] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Oct 17 10:15:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:15:11.243425 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9179207-0f38-4662-900d-764a1507456e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:11.307490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:11.357143 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9179207-0f38-4662-900d-764a1507456e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:11.396544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:11.397205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:11.397432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:11.397701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:11.398425 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:11.398766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:11.400721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:11.409483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 461/927] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Oct 17 10:15:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:11.415626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5c3a-af37-4d61-9b48-1e34c33021fd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:11.534845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5c3a-af37-4d61-9b48-1e34c33021fd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:11.589784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:11.591182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:11.591600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:11.592049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:11.609617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:11.611454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:11.630301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:11.631501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:11.632572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:11.632572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:11.632572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:11.632572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 467/928] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:11.638508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:11.761769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51f38a-221c-4c93-9e91-0d9962e5a6e0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:11.781533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:11.802436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:11.803045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:11.803318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:11.803586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:11.804432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:11.804882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:11.806989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:11.822200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 462/929] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:11.828324 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:11.892407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:11.947833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6cca-23a2-467b-af6d-3d4a685c7fa9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:11.989422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:11.990053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:11.990359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:11.990634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:12.000601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:12.003186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:15:12.017245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:12.023955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:12.025352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 468/930] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:12.026579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:12.163521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f0db1-f71d-476c-b797-b689cb0e1963 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:12.208690 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:12.209697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:12.210064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:12.210345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:12.211101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:12.211393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:12.213518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:12.242356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 463/931] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Oct 17 10:15:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:12.246363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:12.373896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d6499-dbdd-4435-a956-748a24224b0d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:12.384356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:12.411947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a1e77-7eac-449a-8dfe-b4053f0aab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:12.412653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1e77-7eac-449a-8dfe-b4053f0aab08 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:12.413060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1e77-7eac-449a-8dfe-b4053f0aab08 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:12.413347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1e77-7eac-449a-8dfe-b4053f0aab08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:12.505373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a1e77-7eac-449a-8dfe-b4053f0aab08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:12.506872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a1e77-7eac-449a-8dfe-b4053f0aab08 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:12.620785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 469/932] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:15:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:15:12.627251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:12.671620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:12.753164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9e248-e4bd-475a-aac8-46f4f8a110c7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:12.802164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:12.802968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:12.803097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:12.803257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:12.815771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:12.817766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:12.836842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:12.838489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:12.840954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:12.840954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:12.840954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:12.840954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 464/933] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 10:15:12.845365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a1e77-7eac-449a-8dfe-b4053f0aab08 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:12.857798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:12.963942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a1e77-7eac-449a-8dfe-b4053f0aab08 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:13.003892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:13.004499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:13.004777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:13.005049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:13.005683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:13.005986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:13.007379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:13.019434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 470/934] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:15:13.025519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:13.118844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:13.135490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7172547-93ba-430e-8c10-1d95a4d883fe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:13.171963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:13.172836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:13.174243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:13.174611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:13.181199 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:13.182828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:15:13.197072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:13.198168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:13.199306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 465/935] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:13.204325 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:13.278089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:13.314482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9669e-78a3-4f6f-b1d9-fb2f018ad628 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:13.354534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1496a9f-c2ec-40f2-9c76-911da940307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:13.355141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1496a9f-c2ec-40f2-9c76-911da940307a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:13.355478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1496a9f-c2ec-40f2-9c76-911da940307a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:13.355779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1496a9f-c2ec-40f2-9c76-911da940307a None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:13.356497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1496a9f-c2ec-40f2-9c76-911da940307a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:13.356793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1496a9f-c2ec-40f2-9c76-911da940307a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:13.358199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1496a9f-c2ec-40f2-9c76-911da940307a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:13.371631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 471/936] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:13.378606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:13.492508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9669bbe-5ed6-4b62-b9eb-37de42e5aec7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:13.526573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a174e5-fb9a-4915-bd0f-80dce908a95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:13.527225 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a174e5-fb9a-4915-bd0f-80dce908a95d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:13.527505 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a174e5-fb9a-4915-bd0f-80dce908a95d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:13.527779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a174e5-fb9a-4915-bd0f-80dce908a95d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:13.528506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a174e5-fb9a-4915-bd0f-80dce908a95d None admin] RBAC: Adding query filter params (role.id=4045fa40bdd54e9f889b47e1fb2704f5, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=dbb92e22612b400dbf390e149abbb105) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:13.528806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a174e5-fb9a-4915-bd0f-80dce908a95d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:13.530355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a174e5-fb9a-4915-bd0f-80dce908a95d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:13.536266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 466/937] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:15:13 2022] GET /identity/v3/role_assignments?user.id=dbb92e22612b400dbf390e149abbb105&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=4045fa40bdd54e9f889b47e1fb2704f5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:15.842597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66516b-9b95-432a-86d3-bab55001acaa None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:15.843343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66516b-9b95-432a-86d3-bab55001acaa None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:15.843343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66516b-9b95-432a-86d3-bab55001acaa None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:15.845248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 472/938] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:15:15.852276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0dcd0-04ff-4c12-818c-c9f1e8f9a8b8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:15.852760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0dcd0-04ff-4c12-818c-c9f1e8f9a8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:15.853002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0dcd0-04ff-4c12-818c-c9f1e8f9a8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:15.872645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-44f0dcd0-04ff-4c12-818c-c9f1e8f9a8b8 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:15.875057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44f0dcd0-04ff-4c12-818c-c9f1e8f9a8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:15.878133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:15.948027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 467/939] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:15:16.245954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66516b-9b95-432a-86d3-bab55001acaa None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:16.364260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66516b-9b95-432a-86d3-bab55001acaa None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:16.401210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:16.408400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40c25d-8cb1-4e17-bd35-2abdee54d402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:16.409049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c25d-8cb1-4e17-bd35-2abdee54d402 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:16.409367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c25d-8cb1-4e17-bd35-2abdee54d402 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:16.409630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c25d-8cb1-4e17-bd35-2abdee54d402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:16.420474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40c25d-8cb1-4e17-bd35-2abdee54d402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-df40c25d-8cb1-4e17-bd35-2abdee54d402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:15:16.425375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:16.426384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:16.427371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 473/940] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:16.433066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0dcd0-04ff-4c12-818c-c9f1e8f9a8b8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:16.538559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0dcd0-04ff-4c12-818c-c9f1e8f9a8b8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:16.578013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:16.578846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:16.579935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:16.580213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:16.581250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:16.581554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:16.582963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:16.591529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 468/941] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:15:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:15:16.598073 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40c25d-8cb1-4e17-bd35-2abdee54d402 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:16.704519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40c25d-8cb1-4e17-bd35-2abdee54d402 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:16.742643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:16.743469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:16.743771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:16.744211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:16.759361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:16.761868 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:16.779912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:16.781281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:16.782988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:16.782988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:16.782988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:16.782988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 474/942] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:16.788910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:16.792198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:16.901476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a65fd9-f921-482f-ad31-cb3511b92fbd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:16.939131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:16.940577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:16.940853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:16.941158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:16.941813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:16.942113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:16.943637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:16.956562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 469/943] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:16.963063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:17.015629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:17.085653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b8a8f-1899-4d58-9685-b31622d4b195 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:17.129685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:17.130586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:17.131012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:17.131477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:17.142352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:17.144075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:15:17.157546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:17.158742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:17.159904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 475/944] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:17.165552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:17.233492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:17.277515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535951ae-26fc-4793-9f06-0846de4bf67a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:17.316319 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:17.316949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:17.317349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:17.317502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:17.318560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:17.318863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:17.320491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:17.346138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 470/945] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Oct 17 10:15:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:15:17.352786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:17.462013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88865f31-43cd-4d33-aa34-57ed1ef51e22 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:17.463603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:17.499521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:17.500139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:17.500412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:17.500682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:17.511598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:17.513123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:17.535080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:17.536406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:17.537497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:17.537497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:17.537497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:17.537497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 476/946] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:17.542794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:17.659465 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db5525-9e11-40ef-ae19-4f6bce5439da None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:17.685879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:17.706143 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:17.706997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:17.707272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:17.707538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:17.708259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:17.708629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:17.710224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:17.722848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 471/947] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:17.729498 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:17.854586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0618b6c-219a-4554-8aa2-abad19523149 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:17.899908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:17.900571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:17.901055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:17.901526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:17.912449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:17.914651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:15:17.929523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:17.930656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:17.931581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 477/948] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:17.939474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:18.054101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa85072-5cbf-43f7-a44a-0fbaf9456bfc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:18.095196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f413235-93af-4d59-87df-3f7c6faee065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:18.095810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f413235-93af-4d59-87df-3f7c6faee065 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:18.096079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f413235-93af-4d59-87df-3f7c6faee065 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:18.096345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f413235-93af-4d59-87df-3f7c6faee065 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:18.097042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f413235-93af-4d59-87df-3f7c6faee065 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:18.097343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f413235-93af-4d59-87df-3f7c6faee065 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:18.099514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f413235-93af-4d59-87df-3f7c6faee065 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:18.117923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 472/949] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:18.124327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:18.142677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:18.239852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ee2b8-90c8-42ac-b8de-d14c7f36f84b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:18.281414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc159c-0c79-4f8e-a595-46664ac470c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:18.282661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc159c-0c79-4f8e-a595-46664ac470c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:18.283146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc159c-0c79-4f8e-a595-46664ac470c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:18.283646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc159c-0c79-4f8e-a595-46664ac470c3 None admin] PATH_INFO: `/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/dbb92e22612b400dbf390e149abbb105/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:18.286441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc159c-0c79-4f8e-a595-46664ac470c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb915634f9884b49b79a10379df4b0d3, user_id=dbb92e22612b400dbf390e149abbb105, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:18.288302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc159c-0c79-4f8e-a595-46664ac470c3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:18.304085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 478/950] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Oct 17 10:15:18 2022] PUT /identity/v3/projects/eb915634f9884b49b79a10379df4b0d3/users/dbb92e22612b400dbf390e149abbb105/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:15:21.002625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fca1-7a69-4591-818a-764c3283cb13 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:21.003206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fca1-7a69-4591-818a-764c3283cb13 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:21.003206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fca1-7a69-4591-818a-764c3283cb13 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:21.004468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 473/951] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:15:21.009623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693e1db-3eff-4f07-8c90-b757eb4501d8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:21.010083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693e1db-3eff-4f07-8c90-b757eb4501d8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:21.010186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693e1db-3eff-4f07-8c90-b757eb4501d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:21.028711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9693e1db-3eff-4f07-8c90-b757eb4501d8 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:21.030411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9693e1db-3eff-4f07-8c90-b757eb4501d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:21.110943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 479/952] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:15:21.418517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7fca1-7a69-4591-818a-764c3283cb13 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:21.463366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:21.530287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7fca1-7a69-4591-818a-764c3283cb13 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:21.567716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fd68f-afea-477a-ba6f-a6113f95b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:21.568434 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd68f-afea-477a-ba6f-a6113f95b362 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:21.568765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd68f-afea-477a-ba6f-a6113f95b362 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:21.569189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd68f-afea-477a-ba6f-a6113f95b362 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:21.583167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fd68f-afea-477a-ba6f-a6113f95b362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-372fd68f-afea-477a-ba6f-a6113f95b362 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 10:15:21.589006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:21.590174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:21.591268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 474/953] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Oct 17 10:15:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:21.600329 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693e1db-3eff-4f07-8c90-b757eb4501d8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:21.603901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:21.716508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693e1db-3eff-4f07-8c90-b757eb4501d8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:21.763590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:21.764522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:21.764959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:21.765371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:21.767120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:21.767728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:21.769874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:21.782977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 480/954] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Oct 17 10:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:21.791118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fd68f-afea-477a-ba6f-a6113f95b362 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:21.904235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:21.907153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fd68f-afea-477a-ba6f-a6113f95b362 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:21.949921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:21.950579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:21.950855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:21.951127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:21.962552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:21.964130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:21.980325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:21.981492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:21.982737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:21.982737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:21.982737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:21.982737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 475/955] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:21.988557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:22.060992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:22.100592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17094e6-b7ea-4885-bdb3-cf42a509d8a7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:22.139119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:22.139736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:22.140014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:22.140279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:22.140934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:22.141266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:22.142686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:22.155683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 481/956] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:22.162788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:22.267692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e2b05-df79-4edb-8c8c-390e92d8f2c9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:22.303500 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:22.304134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:22.304408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:22.304672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:22.313416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:22.314917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 10:15:22.326943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:22.333045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:22.334357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 476/957] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:22.335241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:22.442484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc7e61-5307-4161-b8eb-4d5145cdfa6d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:22.469136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:22.481223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:22.481831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:22.482100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:22.482392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:22.483110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:22.483409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:22.484825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:22.502246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 482/958] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Oct 17 10:15:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:22.508002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:22.524778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:22.611286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb7fb3-6f20-4897-9f5f-4842386869c2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:22.645100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ee8e2-d494-4490-8c81-3889e4ad07cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:22.645699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee8e2-d494-4490-8c81-3889e4ad07cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:22.645974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee8e2-d494-4490-8c81-3889e4ad07cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:22.646242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee8e2-d494-4490-8c81-3889e4ad07cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:22.710973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ee8e2-d494-4490-8c81-3889e4ad07cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:22.712391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ee8e2-d494-4490-8c81-3889e4ad07cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:22.725344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:22.815680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 477/959] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Oct 17 10:15:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 10:15:22.821404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:22.917528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8315-3fea-430e-841e-f29f1eb7f4c3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:22.950549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:22.951158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:22.951432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:22.951698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:22.961319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:22.962776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 10:15:22.977718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:22.978894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 10:15:22.980187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 10:15:22.980187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 10:15:22.980187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:22.980187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 483/960] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:22.984361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ee8e2-d494-4490-8c81-3889e4ad07cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:23.078200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ee8e2-d494-4490-8c81-3889e4ad07cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:23.090832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:23.112878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:23.113463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:23.113733 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:23.113997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:23.114658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:23.114955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:23.116290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:23.127140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 478/961] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Oct 17 10:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:23.132674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:23.166486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:23.230348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c138cb-e800-48ab-96a5-ee9967abe648 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:23.263110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:23.263712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:23.263983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:23.264247 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:23.270626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:23.272052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 10:15:23.282809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:23.283924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Oct 17 10:15:23.285138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 484/962] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Oct 17 10:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:23.289629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:23.386009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa466a4-eac8-4430-a5c4-89672b9ac2fc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:23.418005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377d40-6672-4130-a653-b288a91a5565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:23.418606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377d40-6672-4130-a653-b288a91a5565 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:23.418878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377d40-6672-4130-a653-b288a91a5565 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:23.419141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377d40-6672-4130-a653-b288a91a5565 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:23.419808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377d40-6672-4130-a653-b288a91a5565 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:23.420109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377d40-6672-4130-a653-b288a91a5565 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:23.421494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377d40-6672-4130-a653-b288a91a5565 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:23.433650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 479/963] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Oct 17 10:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:23.440313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:23.531699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:23.538046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b414b-2f6b-4d78-a682-44ecb9ce3afd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:23.571840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc507e-4da9-4fe0-a792-ae8d28a87d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:23.572242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc507e-4da9-4fe0-a792-ae8d28a87d6e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:23.572511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc507e-4da9-4fe0-a792-ae8d28a87d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:23.572774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc507e-4da9-4fe0-a792-ae8d28a87d6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:23.573481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc507e-4da9-4fe0-a792-ae8d28a87d6e None admin] RBAC: Adding query filter params (role.id=94cb07e34dbe4bcabcfaca043a83f461, scope.project.id=eb915634f9884b49b79a10379df4b0d3, user.id=dbb92e22612b400dbf390e149abbb105) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:23.573785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc507e-4da9-4fe0-a792-ae8d28a87d6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:23.575239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc507e-4da9-4fe0-a792-ae8d28a87d6e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:23.581191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 485/964] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Oct 17 10:15:23 2022] GET /identity/v3/role_assignments?user.id=dbb92e22612b400dbf390e149abbb105&scope.project.id=eb915634f9884b49b79a10379df4b0d3&role.id=94cb07e34dbe4bcabcfaca043a83f461 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:25.716373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babebf7-8b40-4a70-947d-1e807f1ca5ff None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:25.716937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babebf7-8b40-4a70-947d-1e807f1ca5ff None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:25.716937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babebf7-8b40-4a70-947d-1e807f1ca5ff None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:25.718241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 480/965] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:15:25.723755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2985-47ad-46f4-bc35-871bf6a0bb63 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:25.724176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2985-47ad-46f4-bc35-871bf6a0bb63 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:25.724276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2985-47ad-46f4-bc35-871bf6a0bb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:25.743116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-006a2985-47ad-46f4-bc35-871bf6a0bb63 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:25.744429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-006a2985-47ad-46f4-bc35-871bf6a0bb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:25.825259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 486/966] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:15:26.093966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babebf7-8b40-4a70-947d-1e807f1ca5ff None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:26.124436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:26.194139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babebf7-8b40-4a70-947d-1e807f1ca5ff None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:26.227077 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:26.227682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:26.227960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:26.228229 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:26.228802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:26.230153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:26.239349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:26.240602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:26.241757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:26.245275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 481/967] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Oct 17 10:15:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:26.246495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a2985-47ad-46f4-bc35-871bf6a0bb63 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:26.333860 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:26.343509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a2985-47ad-46f4-bc35-871bf6a0bb63 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:26.382899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f184a-ea51-4293-a3e4-014f3dc51b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:26.383902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f184a-ea51-4293-a3e4-014f3dc51b50 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:26.384399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f184a-ea51-4293-a3e4-014f3dc51b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:26.384893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f184a-ea51-4293-a3e4-014f3dc51b50 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:26.386029 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f184a-ea51-4293-a3e4-014f3dc51b50 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:26.386619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f184a-ea51-4293-a3e4-014f3dc51b50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:26.389160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f184a-ea51-4293-a3e4-014f3dc51b50 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:26.395049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 487/968] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Oct 17 10:15:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:26.400732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:26.492859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:26.498805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0d25f-891c-458c-81a7-27996622937f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:26.531033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d3d37-d3e9-4d8f-98f9-0968a73398db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:26.531618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d3d37-d3e9-4d8f-98f9-0968a73398db None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:26.531937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d3d37-d3e9-4d8f-98f9-0968a73398db None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:26.532207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d3d37-d3e9-4d8f-98f9-0968a73398db None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:26.532828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d3d37-d3e9-4d8f-98f9-0968a73398db None admin] RBAC: Adding query filter params (type=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:26.533126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d3d37-d3e9-4d8f-98f9-0968a73398db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:26.534496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d3d37-d3e9-4d8f-98f9-0968a73398db None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:26.538739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 482/969] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Oct 17 10:15:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:28.699242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e53f0-979c-42db-9b38-5f4aeb5dbdfe None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:28.699780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e53f0-979c-42db-9b38-5f4aeb5dbdfe None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:28.699780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e53f0-979c-42db-9b38-5f4aeb5dbdfe None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:28.701082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 488/970] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:15:28.707258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f5811-2009-4fb5-9599-075809b59f92 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:28.707708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f5811-2009-4fb5-9599-075809b59f92 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:28.707810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f5811-2009-4fb5-9599-075809b59f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:28.726263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-999f5811-2009-4fb5-9599-075809b59f92 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:28.727454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-999f5811-2009-4fb5-9599-075809b59f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:28.796766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 483/971] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:15:29.068619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e53f0-979c-42db-9b38-5f4aeb5dbdfe None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:29.168355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e53f0-979c-42db-9b38-5f4aeb5dbdfe None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:29.201077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6a03b-afaa-4311-8aa1-d244cb6e89ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:29.201665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6a03b-afaa-4311-8aa1-d244cb6e89ae None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:29.201936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6a03b-afaa-4311-8aa1-d244cb6e89ae None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:29.202201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6a03b-afaa-4311-8aa1-d244cb6e89ae None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:29.202866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6a03b-afaa-4311-8aa1-d244cb6e89ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:29.204680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6a03b-afaa-4311-8aa1-d244cb6e89ae None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:29.218380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 489/972] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Oct 17 10:15:29 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:15:31.337686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f195b20-4885-4305-af40-dcba86262cc6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:31.338207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f195b20-4885-4305-af40-dcba86262cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:31.338207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f195b20-4885-4305-af40-dcba86262cc6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:31.339473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 484/973] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:15:31.344567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896731a-9e09-4a99-8faa-6889777f8368 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:31.344985 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896731a-9e09-4a99-8faa-6889777f8368 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:31.345092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896731a-9e09-4a99-8faa-6889777f8368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:31.364084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1896731a-9e09-4a99-8faa-6889777f8368 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:31.365257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1896731a-9e09-4a99-8faa-6889777f8368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:31.374282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:31.436908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 490/974] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 10:15:31.704179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f195b20-4885-4305-af40-dcba86262cc6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:31.801085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:31.812637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f195b20-4885-4305-af40-dcba86262cc6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:31.847467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:31.848071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:31.848593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:31.848856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] PATH_INFO: `/v3/services/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:31.849419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:31.850784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:31.859109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:31.861594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:31.862781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 485/975] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Oct 17 10:15:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:31.866887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896731a-9e09-4a99-8faa-6889777f8368 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:31.964188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896731a-9e09-4a99-8faa-6889777f8368 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:31.996915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:31.997500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:31.997771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:31.998036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:31.998676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:31.998972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:32.000845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:32.006252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 491/976] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Oct 17 10:15:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:32.012070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:32.107896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cca004-c30a-47cc-8c2f-fa15a94ef0ab None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:32.139745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268aecc-7e6b-4da8-bfd5-65d8380bc1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:32.140328 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268aecc-7e6b-4da8-bfd5-65d8380bc1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:32.140605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268aecc-7e6b-4da8-bfd5-65d8380bc1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:32.140866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268aecc-7e6b-4da8-bfd5-65d8380bc1c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:32.142083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268aecc-7e6b-4da8-bfd5-65d8380bc1c9 None admin] RBAC: Adding query filter params (interface=public, service_id=4e77fdaf7ad44bbf9922c07fc1d8c6b7, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:32.142420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268aecc-7e6b-4da8-bfd5-65d8380bc1c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:32.143743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268aecc-7e6b-4da8-bfd5-65d8380bc1c9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:32.148467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 486/977] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Oct 17 10:15:32 2022] GET /identity/v3/endpoints?service_id=4e77fdaf7ad44bbf9922c07fc1d8c6b7&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:32.154023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:32.157922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:32.263836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b975865-93b5-45e9-a9fc-b3c42cdd4e62 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:32.295585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232af3b-94ef-4564-a494-578cd460436e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:32.296177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232af3b-94ef-4564-a494-578cd460436e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:32.296452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232af3b-94ef-4564-a494-578cd460436e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:32.296719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232af3b-94ef-4564-a494-578cd460436e None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:32.297824 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232af3b-94ef-4564-a494-578cd460436e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:32.299205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232af3b-94ef-4564-a494-578cd460436e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:32.305666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 492/978] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Oct 17 10:15:32 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:15:34.444086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38397b-4fb2-409e-b486-b9615114fc69 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:34.444612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38397b-4fb2-409e-b486-b9615114fc69 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:34.444612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38397b-4fb2-409e-b486-b9615114fc69 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:34.445872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 487/979] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:15:34.451249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efb5d7-3028-486b-812c-f7ba954b1f96 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:34.451658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efb5d7-3028-486b-812c-f7ba954b1f96 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:34.451790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efb5d7-3028-486b-812c-f7ba954b1f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:34.470369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-03efb5d7-3028-486b-812c-f7ba954b1f96 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:15:34.471517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-03efb5d7-3028-486b-812c-f7ba954b1f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:15:34.530872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:34.537409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 493/980] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:15:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 10:15:34.820504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38397b-4fb2-409e-b486-b9615114fc69 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:34.841430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:34.921673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38397b-4fb2-409e-b486-b9615114fc69 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:34.953794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:34.954440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:34.954718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:34.954985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:34.955552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:34.956885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 10:15:34.965301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 10:15:34.966509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Oct 17 10:15:34.967390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 488/981] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Oct 17 10:15:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 10:15:34.973205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efb5d7-3028-486b-812c-f7ba954b1f96 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:35.069174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efb5d7-3028-486b-812c-f7ba954b1f96 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:35.103104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f1c8e-ce48-4cec-b5a5-26d562d084a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:35.103697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1c8e-ce48-4cec-b5a5-26d562d084a8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:35.103966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1c8e-ce48-4cec-b5a5-26d562d084a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:35.104228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1c8e-ce48-4cec-b5a5-26d562d084a8 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:35.104841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f1c8e-ce48-4cec-b5a5-26d562d084a8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:15:35.105135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f1c8e-ce48-4cec-b5a5-26d562d084a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:35.106515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f1c8e-ce48-4cec-b5a5-26d562d084a8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:35.112046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 494/982] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Oct 17 10:15:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:15:35.117931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:15:35.195950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:35.213364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8226df-b2fa-47db-ad89-6517b670f436 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:15:35.247895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e8e61-9b7a-41f9-8962-9c12f2991ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:15:35.248072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e8e61-9b7a-41f9-8962-9c12f2991ea5 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:15:35.248296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e8e61-9b7a-41f9-8962-9c12f2991ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:15:35.248564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e8e61-9b7a-41f9-8962-9c12f2991ea5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:15:35.249263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e8e61-9b7a-41f9-8962-9c12f2991ea5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:15:35.250809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e8e61-9b7a-41f9-8962-9c12f2991ea5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:15:35.270729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 489/983] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:15:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:15:45.108473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=69 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:15:45.265474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:39.015459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d334c-72a7-4536-8870-3f00cde84980 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:39.016077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d334c-72a7-4536-8870-3f00cde84980 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:39.016077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d334c-72a7-4536-8870-3f00cde84980 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:39.021940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 495/984] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:16:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:39.023288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbffb5-4ea1-4cb6-94f2-7c41ba202c13 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:39.023699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbffb5-4ea1-4cb6-94f2-7c41ba202c13 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:39.023890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbffb5-4ea1-4cb6-94f2-7c41ba202c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:39.120807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-45fbffb5-4ea1-4cb6-94f2-7c41ba202c13 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:39.122992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-45fbffb5-4ea1-4cb6-94f2-7c41ba202c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:39.298983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 490/985] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:16:39.626284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288356be-872e-4034-9881-4539c1d0d855 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:39.626632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288356be-872e-4034-9881-4539c1d0d855 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:39.626862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288356be-872e-4034-9881-4539c1d0d855 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:39.628417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 496/986] 10.222.0.16 () {54 vars in 964 bytes} [Mon Oct 17 10:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:39.634693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483a2d-72a6-43af-9ccb-c5a5c3e385ca None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:39.634972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483a2d-72a6-43af-9ccb-c5a5c3e385ca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:39.635241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483a2d-72a6-43af-9ccb-c5a5c3e385ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:39.725235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-de483a2d-72a6-43af-9ccb-c5a5c3e385ca None None] MFA Rules not processed for user `bd7883e3f6ab4f58969f51ffd439b1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:39.726450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-de483a2d-72a6-43af-9ccb-c5a5c3e385ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:39.750704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:39.818842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 491/987] 10.222.0.16 () {58 vars in 1061 bytes} [Mon Oct 17 10:16:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:16:39.828054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288356be-872e-4034-9881-4539c1d0d855 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:16:39.829847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-288356be-872e-4034-9881-4539c1d0d855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:39.970422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288356be-872e-4034-9881-4539c1d0d855 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:16:40.063207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983e2c4-e47c-4c5a-bd8b-54fed5dc0600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:16:40.064563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983e2c4-e47c-4c5a-bd8b-54fed5dc0600 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:40.064841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983e2c4-e47c-4c5a-bd8b-54fed5dc0600 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:40.065109 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983e2c4-e47c-4c5a-bd8b-54fed5dc0600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:40.066757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7983e2c4-e47c-4c5a-bd8b-54fed5dc0600 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:40.135618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:40.203646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983e2c4-e47c-4c5a-bd8b-54fed5dc0600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:16:40.205110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983e2c4-e47c-4c5a-bd8b-54fed5dc0600 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:16:40.226083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 497/988] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:16:42.758887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686fb0-a0f4-4164-89bc-560442f6d06f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:42.759375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686fb0-a0f4-4164-89bc-560442f6d06f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:42.759478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686fb0-a0f4-4164-89bc-560442f6d06f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:42.760926 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 492/989] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:42.766699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21813c-e223-453b-b87f-02b575eef97d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:42.767115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21813c-e223-453b-b87f-02b575eef97d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:42.767296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21813c-e223-453b-b87f-02b575eef97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:42.842609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3d21813c-e223-453b-b87f-02b575eef97d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:42.843822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3d21813c-e223-453b-b87f-02b575eef97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:42.939128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 498/990] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:16:43.223345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3686fb0-a0f4-4164-89bc-560442f6d06f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:16:43.225127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f3686fb0-a0f4-4164-89bc-560442f6d06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:43.330946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3686fb0-a0f4-4164-89bc-560442f6d06f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:16:43.400969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:43.403436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e65816-06d7-4d82-9f92-7e06fe466e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:16:43.404060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e65816-06d7-4d82-9f92-7e06fe466e4a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:43.404544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e65816-06d7-4d82-9f92-7e06fe466e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:43.405542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e65816-06d7-4d82-9f92-7e06fe466e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:43.407713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-24e65816-06d7-4d82-9f92-7e06fe466e4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:43.515504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e65816-06d7-4d82-9f92-7e06fe466e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:16:43.516922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e65816-06d7-4d82-9f92-7e06fe466e4a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:16:43.535397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 493/991] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:16:45.992175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0e2cf-dbe1-4dbb-bd8d-d1c28cc40b1d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:45.992716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0e2cf-dbe1-4dbb-bd8d-d1c28cc40b1d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:45.992716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0e2cf-dbe1-4dbb-bd8d-d1c28cc40b1d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:45.994031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 499/992] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:45.999921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69cd30-152f-497f-bd27-75ad24637a4f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:46.000350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69cd30-152f-497f-bd27-75ad24637a4f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:46.000465 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69cd30-152f-497f-bd27-75ad24637a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:46.064619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9d69cd30-152f-497f-bd27-75ad24637a4f None None] MFA Rules not processed for user `10a6976357d0445a9fa2891d68878d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:46.065817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9d69cd30-152f-497f-bd27-75ad24637a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:46.117466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:46.162570 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 494/993] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:16:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:16:46.445340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0e2cf-dbe1-4dbb-bd8d-d1c28cc40b1d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:16:46.446942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3da0e2cf-dbe1-4dbb-bd8d-d1c28cc40b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:46.456608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:46.549223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0e2cf-dbe1-4dbb-bd8d-d1c28cc40b1d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:16:46.621239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb75d7-261c-4f0b-9499-46f243d20813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:16:46.621825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb75d7-261c-4f0b-9499-46f243d20813 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:46.622098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb75d7-261c-4f0b-9499-46f243d20813 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:46.622388 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb75d7-261c-4f0b-9499-46f243d20813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:46.624061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a5bb75d7-261c-4f0b-9499-46f243d20813 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:46.684556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:46.730979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb75d7-261c-4f0b-9499-46f243d20813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:16:46.732333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb75d7-261c-4f0b-9499-46f243d20813 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:16:46.750772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 500/994] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:16:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:16:49.440906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aad52-2596-403c-b7b5-f8eb88e7e10d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:49.441417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aad52-2596-403c-b7b5-f8eb88e7e10d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:49.441417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aad52-2596-403c-b7b5-f8eb88e7e10d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:49.442840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 495/995] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:49.447951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945883c1-adef-4aa4-8556-bf0d2e37ecbd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:49.448361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945883c1-adef-4aa4-8556-bf0d2e37ecbd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:49.448497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945883c1-adef-4aa4-8556-bf0d2e37ecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:49.514927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-945883c1-adef-4aa4-8556-bf0d2e37ecbd None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:49.516107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-945883c1-adef-4aa4-8556-bf0d2e37ecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:49.610763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 501/996] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:16:49.895330 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aad52-2596-403c-b7b5-f8eb88e7e10d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:16:49.896843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b1aad52-2596-403c-b7b5-f8eb88e7e10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:49.997618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aad52-2596-403c-b7b5-f8eb88e7e10d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:16:50.070554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15033d77-0e0f-4feb-a824-6522f8b87496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:16:50.071143 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033d77-0e0f-4feb-a824-6522f8b87496 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:50.071411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033d77-0e0f-4feb-a824-6522f8b87496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:50.071676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033d77-0e0f-4feb-a824-6522f8b87496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:50.073241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-15033d77-0e0f-4feb-a824-6522f8b87496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:50.080272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:50.180518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15033d77-0e0f-4feb-a824-6522f8b87496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:16:50.181838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15033d77-0e0f-4feb-a824-6522f8b87496 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:16:50.199650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 496/997] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:16:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:16:53.016621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cd9e1-e2d3-48e3-8bed-5e12ab7ec8af None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:53.017135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cd9e1-e2d3-48e3-8bed-5e12ab7ec8af None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:53.017135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cd9e1-e2d3-48e3-8bed-5e12ab7ec8af None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:53.018432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 502/998] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:53.023809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d439-2808-4a3e-b7cb-c8783830e76c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:53.024259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d439-2808-4a3e-b7cb-c8783830e76c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:53.024488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d439-2808-4a3e-b7cb-c8783830e76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:53.103334 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0623d439-2808-4a3e-b7cb-c8783830e76c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:53.104551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0623d439-2808-4a3e-b7cb-c8783830e76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:53.166037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:53.208893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 497/999] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:16:53.508815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50289015-5837-4a6e-bf74-d51d97c97b82 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:53.509095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50289015-5837-4a6e-bf74-d51d97c97b82 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:53.509386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50289015-5837-4a6e-bf74-d51d97c97b82 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:53.510801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 503/1000] 10.222.0.16 () {54 vars in 964 bytes} [Mon Oct 17 10:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:53.516920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8425-6c87-4bc7-861d-0ff413b8a5e3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:53.517204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8425-6c87-4bc7-861d-0ff413b8a5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:53.517479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8425-6c87-4bc7-861d-0ff413b8a5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:53.595663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ada8425-6c87-4bc7-861d-0ff413b8a5e3 None None] MFA Rules not processed for user `bd7883e3f6ab4f58969f51ffd439b1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:53.596850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ada8425-6c87-4bc7-861d-0ff413b8a5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:53.675315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 498/1001] 10.222.0.16 () {58 vars in 1061 bytes} [Mon Oct 17 10:16:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:16:53.684585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50289015-5837-4a6e-bf74-d51d97c97b82 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:16:53.686980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-50289015-5837-4a6e-bf74-d51d97c97b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:53.706831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:53.792555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50289015-5837-4a6e-bf74-d51d97c97b82 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:16:53.868941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d917a26-ab65-4e52-a5c7-b51ca4ca4361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:16:53.869535 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d917a26-ab65-4e52-a5c7-b51ca4ca4361 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:53.869804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d917a26-ab65-4e52-a5c7-b51ca4ca4361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:53.870069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d917a26-ab65-4e52-a5c7-b51ca4ca4361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:53.871664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d917a26-ab65-4e52-a5c7-b51ca4ca4361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:53.945001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:53.985287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d917a26-ab65-4e52-a5c7-b51ca4ca4361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:16:53.986746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d917a26-ab65-4e52-a5c7-b51ca4ca4361 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:16:54.005424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 504/1002] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:16:56.442255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2815e83-563c-4de8-bfc0-1f297717cd5b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:56.442869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2815e83-563c-4de8-bfc0-1f297717cd5b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:56.442869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2815e83-563c-4de8-bfc0-1f297717cd5b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:56.444201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 499/1003] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:56.449221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da2bbe-b14f-41b0-895b-4b38aedaa92d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:56.449641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da2bbe-b14f-41b0-895b-4b38aedaa92d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:56.449739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da2bbe-b14f-41b0-895b-4b38aedaa92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:56.518996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-75da2bbe-b14f-41b0-895b-4b38aedaa92d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:56.520186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-75da2bbe-b14f-41b0-895b-4b38aedaa92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:56.619473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 505/1004] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:16:56.901009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2815e83-563c-4de8-bfc0-1f297717cd5b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:16:56.902839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c2815e83-563c-4de8-bfc0-1f297717cd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:56.963838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:57.008847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2815e83-563c-4de8-bfc0-1f297717cd5b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:16:57.094571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60830cde-d598-49ff-85f5-320fc3cf7914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:16:57.095169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830cde-d598-49ff-85f5-320fc3cf7914 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:57.095442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830cde-d598-49ff-85f5-320fc3cf7914 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:57.095706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830cde-d598-49ff-85f5-320fc3cf7914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:57.097276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60830cde-d598-49ff-85f5-320fc3cf7914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:57.204875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:16:57.207702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60830cde-d598-49ff-85f5-320fc3cf7914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:16:57.209589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60830cde-d598-49ff-85f5-320fc3cf7914 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:16:57.227640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 500/1005] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:16:59.504087 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438af37-e976-41c4-843d-d53465784b09 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:59.504545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438af37-e976-41c4-843d-d53465784b09 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:59.504647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438af37-e976-41c4-843d-d53465784b09 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:59.506016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 506/1006] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:16:59.511707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d2026-b268-4368-af78-1626d3fb34a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:16:59.512214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d2026-b268-4368-af78-1626d3fb34a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:16:59.512317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d2026-b268-4368-af78-1626d3fb34a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:16:59.580950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-266d2026-b268-4368-af78-1626d3fb34a0 None None] MFA Rules not processed for user `10a6976357d0445a9fa2891d68878d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:16:59.582149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-266d2026-b268-4368-af78-1626d3fb34a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:16:59.678756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 501/1007] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:16:59.969109 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438af37-e976-41c4-843d-d53465784b09 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:16:59.970659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a438af37-e976-41c4-843d-d53465784b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:00.003192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:00.083105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438af37-e976-41c4-843d-d53465784b09 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:00.160486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72317568-9da4-46bb-a358-cf84d7495bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:00.161094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317568-9da4-46bb-a358-cf84d7495bfb service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:00.161368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317568-9da4-46bb-a358-cf84d7495bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:00.161631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317568-9da4-46bb-a358-cf84d7495bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:00.163264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-72317568-9da4-46bb-a358-cf84d7495bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:00.250471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:00.285881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72317568-9da4-46bb-a358-cf84d7495bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:00.287363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72317568-9da4-46bb-a358-cf84d7495bfb service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:00.306845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 507/1008] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:16:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:02.662494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02104cc9-1032-4ca8-9ada-fd14f2c544bd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:02.663062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02104cc9-1032-4ca8-9ada-fd14f2c544bd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:02.663062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02104cc9-1032-4ca8-9ada-fd14f2c544bd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:02.664353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 502/1009] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:02.669573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441f36d-7ec2-4320-a03f-69e8832b208a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:02.670032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441f36d-7ec2-4320-a03f-69e8832b208a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:02.670420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441f36d-7ec2-4320-a03f-69e8832b208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:02.735893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d441f36d-7ec2-4320-a03f-69e8832b208a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:02.737079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d441f36d-7ec2-4320-a03f-69e8832b208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:02.834053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 508/1010] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:03.117715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02104cc9-1032-4ca8-9ada-fd14f2c544bd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:03.119271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-02104cc9-1032-4ca8-9ada-fd14f2c544bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:03.188768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:03.224301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02104cc9-1032-4ca8-9ada-fd14f2c544bd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:03.296930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44f014-1982-44f6-b412-ea11bc39834a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:03.297515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44f014-1982-44f6-b412-ea11bc39834a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:03.297783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44f014-1982-44f6-b412-ea11bc39834a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:03.298045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44f014-1982-44f6-b412-ea11bc39834a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:03.299631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee44f014-1982-44f6-b412-ea11bc39834a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:03.407653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44f014-1982-44f6-b412-ea11bc39834a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:03.408987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44f014-1982-44f6-b412-ea11bc39834a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:03.418769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:03.427796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 503/1011] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:05.615766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbf1c-eeac-42df-8dde-4300b743800f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:05.616309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbf1c-eeac-42df-8dde-4300b743800f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:05.616309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbf1c-eeac-42df-8dde-4300b743800f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:05.617649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 509/1012] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:05.622784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992ebe9-0e16-4630-86b1-ac72cd0da5fb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:05.623798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992ebe9-0e16-4630-86b1-ac72cd0da5fb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:05.624072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992ebe9-0e16-4630-86b1-ac72cd0da5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:05.689917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0992ebe9-0e16-4630-86b1-ac72cd0da5fb None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:05.691111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0992ebe9-0e16-4630-86b1-ac72cd0da5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:05.790624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 504/1013] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:06.078064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfbf1c-eeac-42df-8dde-4300b743800f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:06.079641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-18dfbf1c-eeac-42df-8dde-4300b743800f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:06.120001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:06.182951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfbf1c-eeac-42df-8dde-4300b743800f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:06.256032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd74787-6e4e-4f6a-93aa-5c08c1b64124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:06.256618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd74787-6e4e-4f6a-93aa-5c08c1b64124 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:06.256892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd74787-6e4e-4f6a-93aa-5c08c1b64124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:06.257159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd74787-6e4e-4f6a-93aa-5c08c1b64124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:06.258762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ddd74787-6e4e-4f6a-93aa-5c08c1b64124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:06.341075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:06.366623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd74787-6e4e-4f6a-93aa-5c08c1b64124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:06.368048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd74787-6e4e-4f6a-93aa-5c08c1b64124 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:06.385902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 510/1014] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:09.193395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc18a9-e893-40ff-8d93-ae05e3405a65 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:09.193961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc18a9-e893-40ff-8d93-ae05e3405a65 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:09.193961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc18a9-e893-40ff-8d93-ae05e3405a65 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:09.195293 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 505/1015] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:09.200292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488e9a5-e7c9-4bb4-b0bf-c73f0438dfd9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:09.200738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488e9a5-e7c9-4bb4-b0bf-c73f0438dfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:09.200848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488e9a5-e7c9-4bb4-b0bf-c73f0438dfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:09.266972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a488e9a5-e7c9-4bb4-b0bf-c73f0438dfd9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:09.268200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a488e9a5-e7c9-4bb4-b0bf-c73f0438dfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:09.365046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 511/1016] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:09.646677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc18a9-e893-40ff-8d93-ae05e3405a65 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:09.648244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ebdc18a9-e893-40ff-8d93-ae05e3405a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:09.724838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:09.752604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc18a9-e893-40ff-8d93-ae05e3405a65 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:09.824590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f035a36-d10e-44c0-a698-ae4b53cf2c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:09.825175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035a36-d10e-44c0-a698-ae4b53cf2c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:09.825442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035a36-d10e-44c0-a698-ae4b53cf2c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:09.825703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035a36-d10e-44c0-a698-ae4b53cf2c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:09.827850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f035a36-d10e-44c0-a698-ae4b53cf2c94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:09.935230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f035a36-d10e-44c0-a698-ae4b53cf2c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:09.937117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f035a36-d10e-44c0-a698-ae4b53cf2c94 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:09.955183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 506/1017] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:14.191506 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f70f1-1acb-40f4-bd3d-f56a964ac279 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:14.192062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f70f1-1acb-40f4-bd3d-f56a964ac279 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:14.192062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f70f1-1acb-40f4-bd3d-f56a964ac279 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:14.193414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 512/1018] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:14.198865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ab7b1-d6f0-4b08-be53-01ed6ad440d4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:14.199279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ab7b1-d6f0-4b08-be53-01ed6ad440d4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:14.199412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ab7b1-d6f0-4b08-be53-01ed6ad440d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:14.205087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:14.269018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e83ab7b1-d6f0-4b08-be53-01ed6ad440d4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:14.270146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e83ab7b1-d6f0-4b08-be53-01ed6ad440d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:14.377896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 507/1019] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:14.669347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f70f1-1acb-40f4-bd3d-f56a964ac279 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:14.671503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2a9f70f1-1acb-40f4-bd3d-f56a964ac279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:14.720360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:14.777229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f70f1-1acb-40f4-bd3d-f56a964ac279 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:14.853583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335bb14-6530-4de8-b6da-b87ae185187e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:14.854191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335bb14-6530-4de8-b6da-b87ae185187e service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:14.854506 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335bb14-6530-4de8-b6da-b87ae185187e service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:14.854778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335bb14-6530-4de8-b6da-b87ae185187e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:14.856420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3335bb14-6530-4de8-b6da-b87ae185187e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:14.950082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:14.967355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335bb14-6530-4de8-b6da-b87ae185187e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:14.968787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335bb14-6530-4de8-b6da-b87ae185187e service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:14.987912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 513/1020] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:17.769116 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9c524-5460-4d8a-925d-16aa3df8ccf5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:17.769658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9c524-5460-4d8a-925d-16aa3df8ccf5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:17.769825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9c524-5460-4d8a-925d-16aa3df8ccf5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:17.771561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 508/1021] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:17.777932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df56e8-2344-4601-9f7e-dda073281d0f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:17.778444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df56e8-2344-4601-9f7e-dda073281d0f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:17.778538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df56e8-2344-4601-9f7e-dda073281d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:17.845597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-64df56e8-2344-4601-9f7e-dda073281d0f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:17.846836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-64df56e8-2344-4601-9f7e-dda073281d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:17.948072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 514/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:17:18.237865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9c524-5460-4d8a-925d-16aa3df8ccf5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:18.239443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97f9c524-5460-4d8a-925d-16aa3df8ccf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:18.331231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:18.350232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9c524-5460-4d8a-925d-16aa3df8ccf5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:18.424270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5928d-0a63-460a-b059-28fd44246569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:18.424855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5928d-0a63-460a-b059-28fd44246569 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:18.425125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5928d-0a63-460a-b059-28fd44246569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:18.425388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5928d-0a63-460a-b059-28fd44246569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:18.427050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-83f5928d-0a63-460a-b059-28fd44246569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:18.536241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5928d-0a63-460a-b059-28fd44246569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:18.537697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5928d-0a63-460a-b059-28fd44246569 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:18.556860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 509/1023] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:17:22.739791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f4600-2e63-41e9-82b5-045330f686eb None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:22.740288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f4600-2e63-41e9-82b5-045330f686eb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:22.740403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f4600-2e63-41e9-82b5-045330f686eb None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:22.741731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 515/1024] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:22.747039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd834f6-e214-4a9f-91bc-6e99e98a13e7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:22.747596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd834f6-e214-4a9f-91bc-6e99e98a13e7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:22.747596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd834f6-e214-4a9f-91bc-6e99e98a13e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:22.766268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:22.824438 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5cd834f6-e214-4a9f-91bc-6e99e98a13e7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:22.825679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5cd834f6-e214-4a9f-91bc-6e99e98a13e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:22.931383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 510/1025] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:23.217598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f4600-2e63-41e9-82b5-045330f686eb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:23.219156 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b36f4600-2e63-41e9-82b5-045330f686eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:23.276637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:23.323399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f4600-2e63-41e9-82b5-045330f686eb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:23.396213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3e95b-963d-4867-a146-45dbc636ed08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:23.396821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3e95b-963d-4867-a146-45dbc636ed08 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:23.397093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3e95b-963d-4867-a146-45dbc636ed08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:23.397358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3e95b-963d-4867-a146-45dbc636ed08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:23.398989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b4f3e95b-963d-4867-a146-45dbc636ed08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:23.503695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:23.508054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3e95b-963d-4867-a146-45dbc636ed08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:23.509493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3e95b-963d-4867-a146-45dbc636ed08 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:23.527421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 516/1026] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:17:28.402865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823fd8d-4c1f-4280-9497-2f9cd560673e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:28.403436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823fd8d-4c1f-4280-9497-2f9cd560673e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:28.403436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823fd8d-4c1f-4280-9497-2f9cd560673e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:28.405551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 511/1027] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:28.410970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da89f3-f70a-4baa-97df-9eb85dfdea31 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:28.411810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da89f3-f70a-4baa-97df-9eb85dfdea31 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:28.412289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da89f3-f70a-4baa-97df-9eb85dfdea31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:28.504417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b4da89f3-f70a-4baa-97df-9eb85dfdea31 None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:28.505862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b4da89f3-f70a-4baa-97df-9eb85dfdea31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:28.706493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 517/1028] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:17:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:32.828259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f2e6-73eb-49e2-9103-8e8f0269b078 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:32.828806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f2e6-73eb-49e2-9103-8e8f0269b078 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:32.828806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f2e6-73eb-49e2-9103-8e8f0269b078 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:32.830139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 512/1029] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:32.835444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060cf4-93d9-4e8c-b7ff-a5d0c631874b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:32.835875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060cf4-93d9-4e8c-b7ff-a5d0c631874b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:32.835974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060cf4-93d9-4e8c-b7ff-a5d0c631874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:32.896305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:32.921393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-62060cf4-93d9-4e8c-b7ff-a5d0c631874b None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:32.922793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-62060cf4-93d9-4e8c-b7ff-a5d0c631874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:32.930160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=59 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:33.036335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 518/1030] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:34.588242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2369663-9173-43d3-a66c-53f0e6e79e5e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:34.588779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2369663-9173-43d3-a66c-53f0e6e79e5e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:34.588779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2369663-9173-43d3-a66c-53f0e6e79e5e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:34.590117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 513/1031] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:34.595847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477a5b0-a4ce-4c35-a2a7-cf26868fc30c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:34.596296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477a5b0-a4ce-4c35-a2a7-cf26868fc30c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:34.596409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477a5b0-a4ce-4c35-a2a7-cf26868fc30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:34.673833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2477a5b0-a4ce-4c35-a2a7-cf26868fc30c None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:34.675078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2477a5b0-a4ce-4c35-a2a7-cf26868fc30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:34.717360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:34.795467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 519/1032] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:38.218288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d579-b6b2-4ebf-8f41-0f3e8cfef55f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:38.218788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d579-b6b2-4ebf-8f41-0f3e8cfef55f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:38.218880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d579-b6b2-4ebf-8f41-0f3e8cfef55f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:38.220344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 514/1033] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:38.226053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f264db4-ce1d-42ec-b120-906ae6fbbb23 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:38.226546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f264db4-ce1d-42ec-b120-906ae6fbbb23 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:38.226629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f264db4-ce1d-42ec-b120-906ae6fbbb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:38.305563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6f264db4-ce1d-42ec-b120-906ae6fbbb23 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:38.306759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6f264db4-ce1d-42ec-b120-906ae6fbbb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:38.409191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 520/1034] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:38.712134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2078-c3dc-4153-804b-a16b3460dee6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:38.712513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2078-c3dc-4153-804b-a16b3460dee6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:38.712888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2078-c3dc-4153-804b-a16b3460dee6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:38.714426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 515/1035] 10.222.0.16 () {54 vars in 961 bytes} [Mon Oct 17 10:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:38.726757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7355c6-67d4-4346-a999-1617a83d9c6f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:38.727554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7355c6-67d4-4346-a999-1617a83d9c6f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:38.728459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7355c6-67d4-4346-a999-1617a83d9c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:38.749998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:38.820005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3d7355c6-67d4-4346-a999-1617a83d9c6f None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:38.821185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3d7355c6-67d4-4346-a999-1617a83d9c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:38.952530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 521/1036] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Oct 17 10:17:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:38.960284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c2078-c3dc-4153-804b-a16b3460dee6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:38.961782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1c2c2078-c3dc-4153-804b-a16b3460dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:39.125894 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c2078-c3dc-4153-804b-a16b3460dee6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:39.203192 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:39.247718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0525bb-0250-46ac-bb19-2c796b2563c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:39.248359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0525bb-0250-46ac-bb19-2c796b2563c9 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:39.248638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0525bb-0250-46ac-bb19-2c796b2563c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:39.248841 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0525bb-0250-46ac-bb19-2c796b2563c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:39.250442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1f0525bb-0250-46ac-bb19-2c796b2563c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:39.358294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0525bb-0250-46ac-bb19-2c796b2563c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:39.360041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0525bb-0250-46ac-bb19-2c796b2563c9 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:39.380265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 516/1037] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:41.813185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45721c74-4e9a-411e-8bf3-dfb8e29f77b4 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:41.813185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45721c74-4e9a-411e-8bf3-dfb8e29f77b4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:41.813185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45721c74-4e9a-411e-8bf3-dfb8e29f77b4 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:41.813185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 522/1038] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:41.818560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d39594-460d-4687-bb7d-40b330c58adb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:41.818560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d39594-460d-4687-bb7d-40b330c58adb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:41.819087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d39594-460d-4687-bb7d-40b330c58adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:41.892752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-80d39594-460d-4687-bb7d-40b330c58adb None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:41.894303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-80d39594-460d-4687-bb7d-40b330c58adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:41.933762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:41.998687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 517/1039] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:42.291090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45721c74-4e9a-411e-8bf3-dfb8e29f77b4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:42.292659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-45721c74-4e9a-411e-8bf3-dfb8e29f77b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:42.367032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:42.471857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45721c74-4e9a-411e-8bf3-dfb8e29f77b4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:42.600909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04830c9-9399-4234-8ac4-331399d706f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:42.601513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04830c9-9399-4234-8ac4-331399d706f6 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:42.601786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04830c9-9399-4234-8ac4-331399d706f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:42.602054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04830c9-9399-4234-8ac4-331399d706f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:42.603690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d04830c9-9399-4234-8ac4-331399d706f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:42.627856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:42.698063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04830c9-9399-4234-8ac4-331399d706f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:42.699707 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04830c9-9399-4234-8ac4-331399d706f6 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:42.719409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 523/1040] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:45.228833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0b188-a342-4398-ac33-010e2600c239 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:45.229441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0b188-a342-4398-ac33-010e2600c239 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:45.229441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0b188-a342-4398-ac33-010e2600c239 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:45.230743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 518/1041] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:45.236900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c32a1-37d1-4dfb-b9bc-be8fea537898 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:45.237305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c32a1-37d1-4dfb-b9bc-be8fea537898 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:45.237475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c32a1-37d1-4dfb-b9bc-be8fea537898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:45.306604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-056c32a1-37d1-4dfb-b9bc-be8fea537898 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:45.307811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-056c32a1-37d1-4dfb-b9bc-be8fea537898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:45.416903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:45.430737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 524/1042] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:45.712502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aecd04-ec4d-45ca-b7d9-801dbaea6724 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:45.713261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aecd04-ec4d-45ca-b7d9-801dbaea6724 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:45.713718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aecd04-ec4d-45ca-b7d9-801dbaea6724 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:45.715237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 519/1043] 10.222.0.16 () {54 vars in 961 bytes} [Mon Oct 17 10:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:45.721447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1952e08-ab39-4672-a43b-b83916f9e32f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:45.721832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1952e08-ab39-4672-a43b-b83916f9e32f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:45.722195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1952e08-ab39-4672-a43b-b83916f9e32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:45.805390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d1952e08-ab39-4672-a43b-b83916f9e32f None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:45.806607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d1952e08-ab39-4672-a43b-b83916f9e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:45.927557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 525/1044] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Oct 17 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:45.935986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aecd04-ec4d-45ca-b7d9-801dbaea6724 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:45.937838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d0aecd04-ec4d-45ca-b7d9-801dbaea6724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:46.106690 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aecd04-ec4d-45ca-b7d9-801dbaea6724 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:46.130557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:46.237385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc535b22-74bf-4289-b1c7-3a6b49b31833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:46.238132 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc535b22-74bf-4289-b1c7-3a6b49b31833 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:46.238528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc535b22-74bf-4289-b1c7-3a6b49b31833 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:46.238895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc535b22-74bf-4289-b1c7-3a6b49b31833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:46.240575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cc535b22-74bf-4289-b1c7-3a6b49b31833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:46.348355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc535b22-74bf-4289-b1c7-3a6b49b31833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:46.350350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc535b22-74bf-4289-b1c7-3a6b49b31833 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:46.370737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 520/1045] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:46.395616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d4af-6e0a-4f76-bc4d-2bef3ad07a32 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:46.396123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d4af-6e0a-4f76-bc4d-2bef3ad07a32 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:46.396219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d4af-6e0a-4f76-bc4d-2bef3ad07a32 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:46.397628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 526/1046] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:46.402782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00f553-2f96-47b1-be86-2546c6558726 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:46.404074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00f553-2f96-47b1-be86-2546c6558726 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:46.404074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00f553-2f96-47b1-be86-2546c6558726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:46.438103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:46.479889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fb00f553-2f96-47b1-be86-2546c6558726 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:46.481557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb00f553-2f96-47b1-be86-2546c6558726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:46.615803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 521/1047] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:46.952766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c6abf-2f9a-40c5-b0ff-db2d12e8eb7f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:46.952766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c6abf-2f9a-40c5-b0ff-db2d12e8eb7f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:46.952766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c6abf-2f9a-40c5-b0ff-db2d12e8eb7f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:46.952766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 527/1048] 10.222.0.16 () {54 vars in 963 bytes} [Mon Oct 17 10:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:46.957502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8526a84-df07-463e-b0b6-35b9ace32521 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:46.957786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8526a84-df07-463e-b0b6-35b9ace32521 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:46.958064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8526a84-df07-463e-b0b6-35b9ace32521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:47.041427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e8526a84-df07-463e-b0b6-35b9ace32521 None None] MFA Rules not processed for user `24f1a6c05ed94f5fa81dd355497b6754`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:47.042650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e8526a84-df07-463e-b0b6-35b9ace32521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:47.097874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:47.130604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 522/1049] 10.222.0.16 () {58 vars in 1060 bytes} [Mon Oct 17 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:47.138574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c6abf-2f9a-40c5-b0ff-db2d12e8eb7f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:47.140106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8d4c6abf-2f9a-40c5-b0ff-db2d12e8eb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:47.198092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:47.260701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c6abf-2f9a-40c5-b0ff-db2d12e8eb7f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:47.351874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afe658-c499-4629-a217-aeec7f9a0224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:47.352553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afe658-c499-4629-a217-aeec7f9a0224 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:47.352811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afe658-c499-4629-a217-aeec7f9a0224 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:47.353061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afe658-c499-4629-a217-aeec7f9a0224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:47.354888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b6afe658-c499-4629-a217-aeec7f9a0224 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:47.470286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:47.486516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afe658-c499-4629-a217-aeec7f9a0224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:47.487968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afe658-c499-4629-a217-aeec7f9a0224 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:47.508553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 528/1050] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:48.628857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59da7b5-53d7-4233-9d39-629413b237d1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:48.629440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59da7b5-53d7-4233-9d39-629413b237d1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:48.629440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59da7b5-53d7-4233-9d39-629413b237d1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:48.630922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 523/1051] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:48.635998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8e6fc-cb5f-418f-b8ea-988fb410bf76 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:48.636421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8e6fc-cb5f-418f-b8ea-988fb410bf76 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:48.636527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8e6fc-cb5f-418f-b8ea-988fb410bf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:48.705043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-58f8e6fc-cb5f-418f-b8ea-988fb410bf76 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:48.706254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-58f8e6fc-cb5f-418f-b8ea-988fb410bf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:48.806218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 529/1052] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:49.097885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59da7b5-53d7-4233-9d39-629413b237d1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:49.099722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c59da7b5-53d7-4233-9d39-629413b237d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:49.254010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59da7b5-53d7-4233-9d39-629413b237d1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:49.309936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:49.381674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5409990-cacb-4316-a8c6-1948015befff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:49.382276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5409990-cacb-4316-a8c6-1948015befff service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:49.382578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5409990-cacb-4316-a8c6-1948015befff service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:49.382849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5409990-cacb-4316-a8c6-1948015befff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:49.384507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e5409990-cacb-4316-a8c6-1948015befff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:49.480658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5409990-cacb-4316-a8c6-1948015befff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:49.483171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5409990-cacb-4316-a8c6-1948015befff service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:49.502582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 524/1053] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:51.734282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2451b-6c46-4866-844f-48a6d2b7622d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:51.734864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2451b-6c46-4866-844f-48a6d2b7622d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:51.734864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2451b-6c46-4866-844f-48a6d2b7622d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:51.736548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 530/1054] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:51.742870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3a6c-303a-49ad-951b-be96defb70d9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:51.744022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3a6c-303a-49ad-951b-be96defb70d9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:51.744515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3a6c-303a-49ad-951b-be96defb70d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:51.815957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b5eb3a6c-303a-49ad-951b-be96defb70d9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:51.817491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b5eb3a6c-303a-49ad-951b-be96defb70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:51.826161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:51.924721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 525/1055] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:52.216185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2451b-6c46-4866-844f-48a6d2b7622d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:52.218035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a1b2451b-6c46-4866-844f-48a6d2b7622d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:52.304571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:52.393572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2451b-6c46-4866-844f-48a6d2b7622d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:52.510765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a0227-36a2-41b4-8c2f-66438ffb4a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:52.511374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0227-36a2-41b4-8c2f-66438ffb4a00 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:52.511648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0227-36a2-41b4-8c2f-66438ffb4a00 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:52.511915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0227-36a2-41b4-8c2f-66438ffb4a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:52.514166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3f4a0227-36a2-41b4-8c2f-66438ffb4a00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:52.533820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:52.612710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0227-36a2-41b4-8c2f-66438ffb4a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:52.614314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0227-36a2-41b4-8c2f-66438ffb4a00 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:52.633360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 531/1056] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:17:54.854965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b16394-8038-44dd-97fc-59c98e285363 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:54.855572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b16394-8038-44dd-97fc-59c98e285363 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:54.855572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b16394-8038-44dd-97fc-59c98e285363 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:54.857152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 526/1057] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:54.862075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954a4f-9bf3-443e-af9e-a3f6aea703cc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:54.862800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954a4f-9bf3-443e-af9e-a3f6aea703cc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:54.862800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954a4f-9bf3-443e-af9e-a3f6aea703cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:54.931397 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fc954a4f-9bf3-443e-af9e-a3f6aea703cc None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:54.932433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc954a4f-9bf3-443e-af9e-a3f6aea703cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:55.042898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:55.050469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 532/1058] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:55.409521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b16394-8038-44dd-97fc-59c98e285363 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:55.411045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-96b16394-8038-44dd-97fc-59c98e285363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:55.562579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:55.573942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b16394-8038-44dd-97fc-59c98e285363 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:55.692475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3685f7c-b24f-4ccc-a262-dabe23ae3a20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:55.693216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3685f7c-b24f-4ccc-a262-dabe23ae3a20 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:55.693573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3685f7c-b24f-4ccc-a262-dabe23ae3a20 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:55.693934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3685f7c-b24f-4ccc-a262-dabe23ae3a20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:55.696040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d3685f7c-b24f-4ccc-a262-dabe23ae3a20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:55.804146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3685f7c-b24f-4ccc-a262-dabe23ae3a20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:55.805743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3685f7c-b24f-4ccc-a262-dabe23ae3a20 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:55.816593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:55.826750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 527/1059] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:17:58.134413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60aa9-8a60-4ce5-b73b-ed951ee87efd None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:58.135002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60aa9-8a60-4ce5-b73b-ed951ee87efd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:58.135002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60aa9-8a60-4ce5-b73b-ed951ee87efd None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:58.136342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 533/1060] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:17:58.142874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eae1c-3188-4cb5-9330-8bd2497b0d08 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:58.143360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eae1c-3188-4cb5-9330-8bd2497b0d08 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:58.143673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eae1c-3188-4cb5-9330-8bd2497b0d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:58.223169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bf9eae1c-3188-4cb5-9330-8bd2497b0d08 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:17:58.224658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bf9eae1c-3188-4cb5-9330-8bd2497b0d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:58.335941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 528/1061] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:17:58.623831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60aa9-8a60-4ce5-b73b-ed951ee87efd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:17:58.625351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b2a60aa9-8a60-4ce5-b73b-ed951ee87efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:58.776019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60aa9-8a60-4ce5-b73b-ed951ee87efd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:17:58.845003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:17:58.898658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e750a-87e7-4242-8d01-d885500d08b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:17:58.899203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e750a-87e7-4242-8d01-d885500d08b5 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:17:58.899482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e750a-87e7-4242-8d01-d885500d08b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:17:58.899762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e750a-87e7-4242-8d01-d885500d08b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:17:58.901412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-839e750a-87e7-4242-8d01-d885500d08b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:17:58.999488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e750a-87e7-4242-8d01-d885500d08b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:17:59.001142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e750a-87e7-4242-8d01-d885500d08b5 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:17:59.019052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 534/1062] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:18:01.333797 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b1a6b-5bde-4e57-b3e0-f74c43f01f99 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:01.334360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b1a6b-5bde-4e57-b3e0-f74c43f01f99 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:01.334360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b1a6b-5bde-4e57-b3e0-f74c43f01f99 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:01.335637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 529/1063] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:01.341364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a9567-3fa7-429b-977e-52be1651d1e9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:01.341793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a9567-3fa7-429b-977e-52be1651d1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:01.341899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a9567-3fa7-429b-977e-52be1651d1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:01.409145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d25a9567-3fa7-429b-977e-52be1651d1e9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:01.410376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d25a9567-3fa7-429b-977e-52be1651d1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:01.451604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:01.517931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 535/1064] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:01.807657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b1a6b-5bde-4e57-b3e0-f74c43f01f99 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:01.809340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd9b1a6b-5bde-4e57-b3e0-f74c43f01f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:01.888311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:01.958835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b1a6b-5bde-4e57-b3e0-f74c43f01f99 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:02.070710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3560eae-ae6a-4243-8b98-3dbf533f51c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:02.071231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3560eae-ae6a-4243-8b98-3dbf533f51c1 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:02.071503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3560eae-ae6a-4243-8b98-3dbf533f51c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:02.071769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3560eae-ae6a-4243-8b98-3dbf533f51c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:02.073331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d3560eae-ae6a-4243-8b98-3dbf533f51c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:02.113086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:02.165162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3560eae-ae6a-4243-8b98-3dbf533f51c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:02.166781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3560eae-ae6a-4243-8b98-3dbf533f51c1 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:02.185979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 530/1065] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:02.732482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43a02-8aac-4fe4-9473-5109f590d805 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:02.733065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43a02-8aac-4fe4-9473-5109f590d805 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:02.733065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43a02-8aac-4fe4-9473-5109f590d805 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:02.734369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 536/1066] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:02.739797 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a79c6-0b36-44c6-b8b5-9c1a18c07371 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:02.740252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a79c6-0b36-44c6-b8b5-9c1a18c07371 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:02.740409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a79c6-0b36-44c6-b8b5-9c1a18c07371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:02.808062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-622a79c6-0b36-44c6-b8b5-9c1a18c07371 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:02.809305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-622a79c6-0b36-44c6-b8b5-9c1a18c07371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:02.908939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 531/1067] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:03.210855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c62501-88da-44e4-aeb4-b597fe5f5b49 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:03.211206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c62501-88da-44e4-aeb4-b597fe5f5b49 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:03.211543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c62501-88da-44e4-aeb4-b597fe5f5b49 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:03.212928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 537/1068] 10.222.0.16 () {54 vars in 962 bytes} [Mon Oct 17 10:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:03.219015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0ce04-a215-4549-9baf-b5a906311bd0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:03.219308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0ce04-a215-4549-9baf-b5a906311bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:03.219582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0ce04-a215-4549-9baf-b5a906311bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:03.235433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:03.295511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e7e0ce04-a215-4549-9baf-b5a906311bd0 None None] MFA Rules not processed for user `9ac934ed0122470cb489d6b324c664f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:03.296708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e7e0ce04-a215-4549-9baf-b5a906311bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:03.373031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 532/1069] 10.222.0.16 () {58 vars in 1059 bytes} [Mon Oct 17 10:18:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:03.382125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c62501-88da-44e4-aeb4-b597fe5f5b49 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:03.383745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-14c62501-88da-44e4-aeb4-b597fe5f5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:03.491302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c62501-88da-44e4-aeb4-b597fe5f5b49 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:03.551501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:03.568025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092729d0-24e0-45c2-b9b9-ce2bf0535627 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ac934ed0122470cb489d6b324c664f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:03.568625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092729d0-24e0-45c2-b9b9-ce2bf0535627 service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:03.568900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092729d0-24e0-45c2-b9b9-ce2bf0535627 service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:03.569170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092729d0-24e0-45c2-b9b9-ce2bf0535627 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:03.570783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-092729d0-24e0-45c2-b9b9-ce2bf0535627 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:03.680387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092729d0-24e0-45c2-b9b9-ce2bf0535627 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:03.681746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092729d0-24e0-45c2-b9b9-ce2bf0535627 service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:03.700953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 538/1070] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:04.415014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1474e-e62a-4b74-8449-721b98318440 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:04.415579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1474e-e62a-4b74-8449-721b98318440 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:04.415579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1474e-e62a-4b74-8449-721b98318440 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:04.416886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 533/1071] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:04.422590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef2dfa-3db3-45b0-854e-fcfeb7faf6f6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:04.423070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef2dfa-3db3-45b0-854e-fcfeb7faf6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:04.423162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef2dfa-3db3-45b0-854e-fcfeb7faf6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:04.451291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2092-8375-41df-a246-8e73fad86173 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:04.451579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2092-8375-41df-a246-8e73fad86173 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:04.451850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2092-8375-41df-a246-8e73fad86173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:04.492913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-71ef2dfa-3db3-45b0-854e-fcfeb7faf6f6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:04.494155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-71ef2dfa-3db3-45b0-854e-fcfeb7faf6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:04.521020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3f0b2092-8375-41df-a246-8e73fad86173 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:04.522235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3f0b2092-8375-41df-a246-8e73fad86173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:04.536518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:04.580939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:04.595046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 539/1072] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:04.614099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 534/1073] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:18:04.662969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3165e8-9139-42ef-aaf3-0d92c16f2ebf None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:04.663326 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3165e8-9139-42ef-aaf3-0d92c16f2ebf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:04.663669 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3165e8-9139-42ef-aaf3-0d92c16f2ebf None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:04.665415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 540/1074] 10.222.0.16 () {54 vars in 943 bytes} [Mon Oct 17 10:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:04.672938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e084b8-29eb-461b-adba-736e87811bfc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:04.673286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e084b8-29eb-461b-adba-736e87811bfc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:04.673629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e084b8-29eb-461b-adba-736e87811bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:04.755125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d9e084b8-29eb-461b-adba-736e87811bfc None None] MFA Rules not processed for user `cf85319814ff4478b40b8a2e28e054ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:04.756629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9e084b8-29eb-461b-adba-736e87811bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:04.874576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 535/1075] 10.222.0.16 () {58 vars in 1040 bytes} [Mon Oct 17 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:18:04.883851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3165e8-9139-42ef-aaf3-0d92c16f2ebf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:04.885389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bf3165e8-9139-42ef-aaf3-0d92c16f2ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:04.888777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e084b8-29eb-461b-adba-736e87811bfc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:04.889929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9e084b8-29eb-461b-adba-736e87811bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:04.923615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:04.991159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e084b8-29eb-461b-adba-736e87811bfc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:05.026389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3165e8-9139-42ef-aaf3-0d92c16f2ebf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:05.063915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:05.114069 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4020d-92a7-414a-8641-721386efb2ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:05.114706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4020d-92a7-414a-8641-721386efb2ad service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:05.115010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4020d-92a7-414a-8641-721386efb2ad service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:05.115285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4020d-92a7-414a-8641-721386efb2ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:05.116911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d8c4020d-92a7-414a-8641-721386efb2ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:05.136447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6c42a-a45c-4977-a14b-05c096375839 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:05.137019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c42a-a45c-4977-a14b-05c096375839 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:05.137360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c42a-a45c-4977-a14b-05c096375839 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:05.137634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c42a-a45c-4977-a14b-05c096375839 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:05.139295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f9d6c42a-a45c-4977-a14b-05c096375839 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:05.160011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:05.208935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4020d-92a7-414a-8641-721386efb2ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:05.210543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4020d-92a7-414a-8641-721386efb2ad service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:05.225608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6c42a-a45c-4977-a14b-05c096375839 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:05.227228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6c42a-a45c-4977-a14b-05c096375839 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:05.229643 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 536/1076] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:18:05.236516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 541/1077] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:18:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:18:07.515171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367aebc-7538-49ba-85d2-f8b687891c98 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:07.515771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367aebc-7538-49ba-85d2-f8b687891c98 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:07.515771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367aebc-7538-49ba-85d2-f8b687891c98 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:07.517170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 537/1078] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:07.522729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c586b-f219-477d-a27f-c81a89d9b0c2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:07.523569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c586b-f219-477d-a27f-c81a89d9b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:07.524067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c586b-f219-477d-a27f-c81a89d9b0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:07.599224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a53c586b-f219-477d-a27f-c81a89d9b0c2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:07.600417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a53c586b-f219-477d-a27f-c81a89d9b0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:07.619460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:07.718501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 542/1079] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:08.011121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367aebc-7538-49ba-85d2-f8b687891c98 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:08.013448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4367aebc-7538-49ba-85d2-f8b687891c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:08.188011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367aebc-7538-49ba-85d2-f8b687891c98 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:08.211283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:08.314634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace624a0-7455-4a09-93c7-a0b463c7751f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:08.318174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace624a0-7455-4a09-93c7-a0b463c7751f service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:08.318479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace624a0-7455-4a09-93c7-a0b463c7751f service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:08.318747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace624a0-7455-4a09-93c7-a0b463c7751f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:08.320415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ace624a0-7455-4a09-93c7-a0b463c7751f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:08.428958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace624a0-7455-4a09-93c7-a0b463c7751f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:08.430577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace624a0-7455-4a09-93c7-a0b463c7751f service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:08.450184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 538/1080] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:10.698167 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e87e6-64c1-4f60-ad1c-874b9a5584ef None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:10.698746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e87e6-64c1-4f60-ad1c-874b9a5584ef None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:10.698746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e87e6-64c1-4f60-ad1c-874b9a5584ef None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:10.700137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 543/1081] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:10.705536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb75eb-81e8-4604-92cb-5dfbcba69d17 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:10.705961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb75eb-81e8-4604-92cb-5dfbcba69d17 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:10.706052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb75eb-81e8-4604-92cb-5dfbcba69d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:10.739279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:10.780469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-52fb75eb-81e8-4604-92cb-5dfbcba69d17 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:10.781843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-52fb75eb-81e8-4604-92cb-5dfbcba69d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:10.890526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 539/1082] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:11.197371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e87e6-64c1-4f60-ad1c-874b9a5584ef None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:11.198933 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0f2e87e6-64c1-4f60-ad1c-874b9a5584ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:11.376060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e87e6-64c1-4f60-ad1c-874b9a5584ef None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:11.378861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:11.510456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35e3ac-c8af-477e-8235-71c65d02498a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:11.511187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e3ac-c8af-477e-8235-71c65d02498a service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:11.511542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e3ac-c8af-477e-8235-71c65d02498a service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:11.511961 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e3ac-c8af-477e-8235-71c65d02498a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:11.514158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6c35e3ac-c8af-477e-8235-71c65d02498a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:11.616231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35e3ac-c8af-477e-8235-71c65d02498a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:11.620574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35e3ac-c8af-477e-8235-71c65d02498a service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:11.638501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 544/1083] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:14.055558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fedd25-9695-474d-ae24-7d7285126177 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:14.056137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fedd25-9695-474d-ae24-7d7285126177 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:14.056137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fedd25-9695-474d-ae24-7d7285126177 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:14.057542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 540/1084] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:14.063096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca50de-4dc8-41e5-8e51-b4e9b167171f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:14.063578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca50de-4dc8-41e5-8e51-b4e9b167171f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:14.063670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca50de-4dc8-41e5-8e51-b4e9b167171f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:14.076638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:14.145126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-01ca50de-4dc8-41e5-8e51-b4e9b167171f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:14.146464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-01ca50de-4dc8-41e5-8e51-b4e9b167171f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:14.264460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 545/1085] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:14.556459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fedd25-9695-474d-ae24-7d7285126177 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:14.557998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-18fedd25-9695-474d-ae24-7d7285126177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:14.669825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:14.724020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fedd25-9695-474d-ae24-7d7285126177 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:14.854240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e771c-dfe9-4e5b-b214-a6bd9f5247c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:14.854876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e771c-dfe9-4e5b-b214-a6bd9f5247c0 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:14.855146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e771c-dfe9-4e5b-b214-a6bd9f5247c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:14.855444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e771c-dfe9-4e5b-b214-a6bd9f5247c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:14.857066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0a6e771c-dfe9-4e5b-b214-a6bd9f5247c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:14.932816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:14.958934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e771c-dfe9-4e5b-b214-a6bd9f5247c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:14.960520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e771c-dfe9-4e5b-b214-a6bd9f5247c0 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:14.982315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 541/1086] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:24.262266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=62 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:24.979406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=16 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:28.018742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04c366-3018-4126-b50b-e37b6d30f4a3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:28.019331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04c366-3018-4126-b50b-e37b6d30f4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:28.019331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04c366-3018-4126-b50b-e37b6d30f4a3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:28.020657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 546/1087] 10.222.0.16 () {54 vars in 918 bytes} [Mon Oct 17 10:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:28.025625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaebe4-3c02-46dd-871d-b04c03bdcb9f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:28.026041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaebe4-3c02-46dd-871d-b04c03bdcb9f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:28.026205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaebe4-3c02-46dd-871d-b04c03bdcb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:28.103000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8afaebe4-3c02-46dd-871d-b04c03bdcb9f None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:28.104173 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8afaebe4-3c02-46dd-871d-b04c03bdcb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:28.209208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 542/1088] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Oct 17 10:18:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:28.224139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fb9f1-3d37-4b70-8cc8-304ade334e36 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:28.224417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fb9f1-3d37-4b70-8cc8-304ade334e36 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:28.224681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fb9f1-3d37-4b70-8cc8-304ade334e36 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:28.226022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 547/1089] 10.222.0.16 () {54 vars in 918 bytes} [Mon Oct 17 10:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:28.231478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49529b18-8b5f-4d37-9344-0dab9031c4d7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:28.231761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49529b18-8b5f-4d37-9344-0dab9031c4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:28.232059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49529b18-8b5f-4d37-9344-0dab9031c4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:28.313611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:28.316978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-49529b18-8b5f-4d37-9344-0dab9031c4d7 None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:28.318433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-49529b18-8b5f-4d37-9344-0dab9031c4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:28.431685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 543/1090] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Oct 17 10:18:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:31.008524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d567a1-028f-4c31-b60e-da130aaa9d21 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:31.009097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d567a1-028f-4c31-b60e-da130aaa9d21 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:31.009097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d567a1-028f-4c31-b60e-da130aaa9d21 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:31.010416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 548/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:31.016208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a954d1-ed63-42c6-990a-a406775f5405 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:31.016642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a954d1-ed63-42c6-990a-a406775f5405 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:31.016747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a954d1-ed63-42c6-990a-a406775f5405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:31.082488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c0a954d1-ed63-42c6-990a-a406775f5405 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:31.083666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c0a954d1-ed63-42c6-990a-a406775f5405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:31.138586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:31.179922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 544/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:31.471720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d567a1-028f-4c31-b60e-da130aaa9d21 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:31.473233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-74d567a1-028f-4c31-b60e-da130aaa9d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:31.618163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d567a1-028f-4c31-b60e-da130aaa9d21 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:31.691346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:31.732107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ea5b6-3e8d-4e94-aa62-3983d820849f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:31.732635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ea5b6-3e8d-4e94-aa62-3983d820849f service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:31.732908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ea5b6-3e8d-4e94-aa62-3983d820849f service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:31.733176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ea5b6-3e8d-4e94-aa62-3983d820849f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:31.734861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-397ea5b6-3e8d-4e94-aa62-3983d820849f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:31.822802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ea5b6-3e8d-4e94-aa62-3983d820849f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:31.824477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ea5b6-3e8d-4e94-aa62-3983d820849f service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:31.841794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 549/1093] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:32.970681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330ea08-a21e-49b0-a02e-071a30d4e5b0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:32.971268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330ea08-a21e-49b0-a02e-071a30d4e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:32.971268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330ea08-a21e-49b0-a02e-071a30d4e5b0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:32.972568 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 545/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:32.978876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539fe9c-f140-4ef1-ace6-91373cd1e8d6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:32.979315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539fe9c-f140-4ef1-ace6-91373cd1e8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:32.979437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539fe9c-f140-4ef1-ace6-91373cd1e8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:33.055429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f539fe9c-f140-4ef1-ace6-91373cd1e8d6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:33.056613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f539fe9c-f140-4ef1-ace6-91373cd1e8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:33.084202 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:33.154771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 550/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:33.303473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330ea08-a21e-49b0-a02e-071a30d4e5b0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:33.304974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3330ea08-a21e-49b0-a02e-071a30d4e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:33.447214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330ea08-a21e-49b0-a02e-071a30d4e5b0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:33.486579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:33.560809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86944743-f114-4cb3-b19c-b71b51f2e865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:33.561403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86944743-f114-4cb3-b19c-b71b51f2e865 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:33.561675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86944743-f114-4cb3-b19c-b71b51f2e865 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:33.561940 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86944743-f114-4cb3-b19c-b71b51f2e865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:33.563530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86944743-f114-4cb3-b19c-b71b51f2e865 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:33.652536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86944743-f114-4cb3-b19c-b71b51f2e865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:33.654104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86944743-f114-4cb3-b19c-b71b51f2e865 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:33.672196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 546/1096] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:34.799836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b6b9a-be48-4fa1-84f6-6ff73d5d01c3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:34.800414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b6b9a-be48-4fa1-84f6-6ff73d5d01c3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:34.800414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b6b9a-be48-4fa1-84f6-6ff73d5d01c3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:34.801794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 551/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:34.807857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed514-415d-454a-ac15-57699de8f958 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:34.809076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed514-415d-454a-ac15-57699de8f958 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:34.809194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed514-415d-454a-ac15-57699de8f958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:34.863173 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:34.885684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d0bed514-415d-454a-ac15-57699de8f958 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:34.887004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d0bed514-415d-454a-ac15-57699de8f958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:34.983480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 547/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:18:35.134233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b6b9a-be48-4fa1-84f6-6ff73d5d01c3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:35.135761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f87b6b9a-be48-4fa1-84f6-6ff73d5d01c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:35.285057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b6b9a-be48-4fa1-84f6-6ff73d5d01c3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:35.297110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:35.398819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465a949-41d8-4dad-80f3-dc9b8b12601a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:35.399357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a949-41d8-4dad-80f3-dc9b8b12601a service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:35.399627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a949-41d8-4dad-80f3-dc9b8b12601a service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:35.399949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a949-41d8-4dad-80f3-dc9b8b12601a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:35.401519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2465a949-41d8-4dad-80f3-dc9b8b12601a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:35.490881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465a949-41d8-4dad-80f3-dc9b8b12601a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:35.492477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465a949-41d8-4dad-80f3-dc9b8b12601a service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:35.510107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 552/1099] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:36.624706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981e9a0-0662-4649-9c1f-7aa666649b00 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:36.625257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981e9a0-0662-4649-9c1f-7aa666649b00 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:36.625383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981e9a0-0662-4649-9c1f-7aa666649b00 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:36.626684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 548/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:36.632991 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367fbbc-56b0-4a6a-9335-f95943b48d3b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:36.633422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367fbbc-56b0-4a6a-9335-f95943b48d3b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:36.633537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367fbbc-56b0-4a6a-9335-f95943b48d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:36.653740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:36.710694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8367fbbc-56b0-4a6a-9335-f95943b48d3b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:36.711885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8367fbbc-56b0-4a6a-9335-f95943b48d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:36.806732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 553/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:36.956030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981e9a0-0662-4649-9c1f-7aa666649b00 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:36.957462 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a981e9a0-0662-4649-9c1f-7aa666649b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:37.064238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:37.100675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981e9a0-0662-4649-9c1f-7aa666649b00 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:37.211080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1af8a-b82f-4264-baec-1725b1c46de2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:37.211571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1af8a-b82f-4264-baec-1725b1c46de2 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:37.211838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1af8a-b82f-4264-baec-1725b1c46de2 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:37.212098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1af8a-b82f-4264-baec-1725b1c46de2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:37.213644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-83b1af8a-b82f-4264-baec-1725b1c46de2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:37.288642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:37.301929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1af8a-b82f-4264-baec-1725b1c46de2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:37.303538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1af8a-b82f-4264-baec-1725b1c46de2 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:37.321801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 549/1102] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:38.438175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6059a54-d63b-4897-a4c3-cf47a1b1c9bd None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:38.438741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6059a54-d63b-4897-a4c3-cf47a1b1c9bd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:38.438741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6059a54-d63b-4897-a4c3-cf47a1b1c9bd None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:38.440039 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 554/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:38.446255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a654-67d9-4b3b-a4f5-3277db7ed302 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:38.446749 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a654-67d9-4b3b-a4f5-3277db7ed302 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:38.446847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a654-67d9-4b3b-a4f5-3277db7ed302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:38.521023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b5d0a654-67d9-4b3b-a4f5-3277db7ed302 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:38.522194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b5d0a654-67d9-4b3b-a4f5-3277db7ed302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:38.616121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 550/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:38.769922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6059a54-d63b-4897-a4c3-cf47a1b1c9bd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:38.771485 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e6059a54-d63b-4897-a4c3-cf47a1b1c9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:38.856815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:38.920337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6059a54-d63b-4897-a4c3-cf47a1b1c9bd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:39.032163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d724f-1b7b-48ac-97a7-3b973f3d3cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:39.032688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d724f-1b7b-48ac-97a7-3b973f3d3cbf service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:39.032962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d724f-1b7b-48ac-97a7-3b973f3d3cbf service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:39.033229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d724f-1b7b-48ac-97a7-3b973f3d3cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:39.034839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-921d724f-1b7b-48ac-97a7-3b973f3d3cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:39.083808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:39.126703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d724f-1b7b-48ac-97a7-3b973f3d3cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:39.128311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d724f-1b7b-48ac-97a7-3b973f3d3cbf service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:39.145735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 555/1105] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:40.206555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94ebac-5e0c-4513-8967-d3d7f796efdc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:40.207076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94ebac-5e0c-4513-8967-d3d7f796efdc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:40.207076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94ebac-5e0c-4513-8967-d3d7f796efdc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:40.208419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 551/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:40.215436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f2aa9-2bda-4d6f-a66f-a7ac5d28d685 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:40.215876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f2aa9-2bda-4d6f-a66f-a7ac5d28d685 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:40.215968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f2aa9-2bda-4d6f-a66f-a7ac5d28d685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:40.291978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bb6f2aa9-2bda-4d6f-a66f-a7ac5d28d685 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:40.294126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bb6f2aa9-2bda-4d6f-a66f-a7ac5d28d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:40.389656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 556/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:40.539577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94ebac-5e0c-4513-8967-d3d7f796efdc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:40.541083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-df94ebac-5e0c-4513-8967-d3d7f796efdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:40.593284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:40.690430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94ebac-5e0c-4513-8967-d3d7f796efdc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:40.802233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6145e0c-1769-4b6c-83c6-ac2cbc256c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:40.802858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145e0c-1769-4b6c-83c6-ac2cbc256c0b service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:40.803131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145e0c-1769-4b6c-83c6-ac2cbc256c0b service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:40.803399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145e0c-1769-4b6c-83c6-ac2cbc256c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:40.804982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e6145e0c-1769-4b6c-83c6-ac2cbc256c0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:40.812470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:40.893402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6145e0c-1769-4b6c-83c6-ac2cbc256c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:40.895001 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6145e0c-1769-4b6c-83c6-ac2cbc256c0b service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:40.913007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 552/1108] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:41.963699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b965-c9da-46ae-b554-7a2127ecbb60 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:41.964804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b965-c9da-46ae-b554-7a2127ecbb60 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:41.964904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b965-c9da-46ae-b554-7a2127ecbb60 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:41.966257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 557/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:41.972484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641a945-abd3-4fa1-8bf9-1235ad4e7e72 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:41.972928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641a945-abd3-4fa1-8bf9-1235ad4e7e72 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:41.973032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641a945-abd3-4fa1-8bf9-1235ad4e7e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:42.047772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f641a945-abd3-4fa1-8bf9-1235ad4e7e72 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:42.048939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f641a945-abd3-4fa1-8bf9-1235ad4e7e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:42.116515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:42.145409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 553/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:42.295313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1b965-c9da-46ae-b554-7a2127ecbb60 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:42.296815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4ad1b965-c9da-46ae-b554-7a2127ecbb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:42.314123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:42.445668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1b965-c9da-46ae-b554-7a2127ecbb60 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:42.531547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:42.556364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf75e1-46ac-4f60-9f42-46fe7ce1df72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:42.556899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf75e1-46ac-4f60-9f42-46fe7ce1df72 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:42.557171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf75e1-46ac-4f60-9f42-46fe7ce1df72 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:42.557434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf75e1-46ac-4f60-9f42-46fe7ce1df72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:42.559761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6cbf75e1-46ac-4f60-9f42-46fe7ce1df72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:42.648476 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf75e1-46ac-4f60-9f42-46fe7ce1df72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:42.650048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf75e1-46ac-4f60-9f42-46fe7ce1df72 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:42.667707 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 558/1111] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:43.727116 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954bf0-6fd4-4d49-84fb-40c5c0696ffb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:43.727706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954bf0-6fd4-4d49-84fb-40c5c0696ffb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:43.727706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954bf0-6fd4-4d49-84fb-40c5c0696ffb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:43.729330 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 554/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:43.738021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e8d1d-b48a-485d-aff4-39202f18ca0e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:43.738539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e8d1d-b48a-485d-aff4-39202f18ca0e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:43.738727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e8d1d-b48a-485d-aff4-39202f18ca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:43.814113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fc3e8d1d-b48a-485d-aff4-39202f18ca0e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:43.815358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc3e8d1d-b48a-485d-aff4-39202f18ca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:43.858183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:43.912840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 559/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:44.062211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a954bf0-6fd4-4d49-84fb-40c5c0696ffb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:44.063745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5a954bf0-6fd4-4d49-84fb-40c5c0696ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:44.212306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a954bf0-6fd4-4d49-84fb-40c5c0696ffb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:44.268167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:44.326414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ddecd-7e6f-415e-a35d-7ce745880f2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:44.326959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ddecd-7e6f-415e-a35d-7ce745880f2a service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:44.327226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ddecd-7e6f-415e-a35d-7ce745880f2a service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:44.327497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ddecd-7e6f-415e-a35d-7ce745880f2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:44.329072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef0ddecd-7e6f-415e-a35d-7ce745880f2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:44.416376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ddecd-7e6f-415e-a35d-7ce745880f2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:44.418480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ddecd-7e6f-415e-a35d-7ce745880f2a service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:44.436561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 555/1114] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:45.492467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899140e-2f79-4f5f-bfb3-20c0d42922e2 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:45.493076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899140e-2f79-4f5f-bfb3-20c0d42922e2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:45.493076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899140e-2f79-4f5f-bfb3-20c0d42922e2 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:45.494406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 560/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:45.501041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4997be-6f30-444e-afb5-56105f299f74 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:45.501488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4997be-6f30-444e-afb5-56105f299f74 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:45.501594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4997be-6f30-444e-afb5-56105f299f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:45.578832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7f4997be-6f30-444e-afb5-56105f299f74 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:45.580016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7f4997be-6f30-444e-afb5-56105f299f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:45.581982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:45.676580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 556/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:45.830266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899140e-2f79-4f5f-bfb3-20c0d42922e2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:45.831807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6899140e-2f79-4f5f-bfb3-20c0d42922e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:45.976772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899140e-2f79-4f5f-bfb3-20c0d42922e2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:46.004640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:46.090566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa90903-1c65-458a-adca-69e5d41739e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:46.091113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa90903-1c65-458a-adca-69e5d41739e3 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:46.091388 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa90903-1c65-458a-adca-69e5d41739e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:46.091655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa90903-1c65-458a-adca-69e5d41739e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:46.093228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3fa90903-1c65-458a-adca-69e5d41739e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:46.183804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa90903-1c65-458a-adca-69e5d41739e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:46.185395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa90903-1c65-458a-adca-69e5d41739e3 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:46.202934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 561/1117] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:48.266892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718b04-ef6e-4f86-829a-767675399157 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:48.267466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718b04-ef6e-4f86-829a-767675399157 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:48.267466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718b04-ef6e-4f86-829a-767675399157 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:48.268807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 557/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:48.274131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10713b0-cc92-4098-b851-bb60bd67a5b6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:48.274591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10713b0-cc92-4098-b851-bb60bd67a5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:48.274729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10713b0-cc92-4098-b851-bb60bd67a5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:48.316140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:48.342429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a10713b0-cc92-4098-b851-bb60bd67a5b6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:48.343660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a10713b0-cc92-4098-b851-bb60bd67a5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:48.439247 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 562/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:48.726598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7718b04-ef6e-4f86-829a-767675399157 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:48.728119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c7718b04-ef6e-4f86-829a-767675399157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:48.845441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7718b04-ef6e-4f86-829a-767675399157 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:48.877594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:48.919306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050dbc1-fe6e-4858-bb0a-a0cca86c39cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:48.919899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050dbc1-fe6e-4858-bb0a-a0cca86c39cb service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:48.920172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050dbc1-fe6e-4858-bb0a-a0cca86c39cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:48.920434 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050dbc1-fe6e-4858-bb0a-a0cca86c39cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:48.922007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a050dbc1-fe6e-4858-bb0a-a0cca86c39cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:49.027985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050dbc1-fe6e-4858-bb0a-a0cca86c39cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:49.029402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050dbc1-fe6e-4858-bb0a-a0cca86c39cb service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:49.047478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 558/1120] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:51.432327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703e534-b7b5-42f7-a968-3093c787fc28 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:51.433378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703e534-b7b5-42f7-a968-3093c787fc28 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:51.433378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703e534-b7b5-42f7-a968-3093c787fc28 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:51.434225 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 563/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:51.439645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8c45-4d97-44e4-a4da-802b95d02cc3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:51.440085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8c45-4d97-44e4-a4da-802b95d02cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:51.440188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8c45-4d97-44e4-a4da-802b95d02cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:51.506048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b1ac8c45-4d97-44e4-a4da-802b95d02cc3 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:51.507268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b1ac8c45-4d97-44e4-a4da-802b95d02cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:51.519927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:51.605813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 559/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:51.889301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703e534-b7b5-42f7-a968-3093c787fc28 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:51.890864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8703e534-b7b5-42f7-a968-3093c787fc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:51.994173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703e534-b7b5-42f7-a968-3093c787fc28 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:52.007318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:52.068959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1cec8-b6da-4d4e-9e14-325b3fb9f82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:52.069558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1cec8-b6da-4d4e-9e14-325b3fb9f82c service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:52.069831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1cec8-b6da-4d4e-9e14-325b3fb9f82c service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:52.070094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1cec8-b6da-4d4e-9e14-325b3fb9f82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:52.071714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9ec1cec8-b6da-4d4e-9e14-325b3fb9f82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:52.181523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1cec8-b6da-4d4e-9e14-325b3fb9f82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:52.183668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1cec8-b6da-4d4e-9e14-325b3fb9f82c service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:52.201863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 564/1123] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:54.642785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa2ad-11c3-4892-9c72-75f0fea38534 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:54.643400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa2ad-11c3-4892-9c72-75f0fea38534 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:54.643400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa2ad-11c3-4892-9c72-75f0fea38534 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:54.644673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 560/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:54.649952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33390a99-2081-417c-a7cf-dac5b9e0c54b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:54.650384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33390a99-2081-417c-a7cf-dac5b9e0c54b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:54.650563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33390a99-2081-417c-a7cf-dac5b9e0c54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:54.698257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:54.717256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-33390a99-2081-417c-a7cf-dac5b9e0c54b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:54.718533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-33390a99-2081-417c-a7cf-dac5b9e0c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:54.815383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 565/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:55.106553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aa2ad-11c3-4892-9c72-75f0fea38534 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:55.108078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9f4aa2ad-11c3-4892-9c72-75f0fea38534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:55.210172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aa2ad-11c3-4892-9c72-75f0fea38534 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:55.252084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:55.286959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a132631-7bf2-42bc-8fb2-aba3becc80b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:55.287545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132631-7bf2-42bc-8fb2-aba3becc80b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:55.287839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132631-7bf2-42bc-8fb2-aba3becc80b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:55.288104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132631-7bf2-42bc-8fb2-aba3becc80b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:55.289672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6a132631-7bf2-42bc-8fb2-aba3becc80b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:55.397181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a132631-7bf2-42bc-8fb2-aba3becc80b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:55.398544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a132631-7bf2-42bc-8fb2-aba3becc80b5 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:55.416650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 561/1126] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:18:57.663896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0c445-91b9-45cb-b634-d5631d44d386 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:57.664474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0c445-91b9-45cb-b634-d5631d44d386 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:57.664474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0c445-91b9-45cb-b634-d5631d44d386 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:57.665754 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 566/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:18:57.670983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c35a0-a598-4be9-9b55-5a80156634ee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:57.671403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c35a0-a598-4be9-9b55-5a80156634ee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:57.671538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c35a0-a598-4be9-9b55-5a80156634ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:57.740137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e47c35a0-a598-4be9-9b55-5a80156634ee None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:18:57.741348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e47c35a0-a598-4be9-9b55-5a80156634ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:57.764316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:57.840796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 562/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:18:58.126233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0c445-91b9-45cb-b634-d5631d44d386 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:18:58.127831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-53c0c445-91b9-45cb-b634-d5631d44d386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:58.231649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0c445-91b9-45cb-b634-d5631d44d386 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:18:58.251854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:18:58.305848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163702e-b8f7-448b-bab7-2d14f314189f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:18:58.306508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163702e-b8f7-448b-bab7-2d14f314189f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:18:58.306791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163702e-b8f7-448b-bab7-2d14f314189f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:18:58.307059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163702e-b8f7-448b-bab7-2d14f314189f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:18:58.308857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0163702e-b8f7-448b-bab7-2d14f314189f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:18:58.419021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163702e-b8f7-448b-bab7-2d14f314189f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:18:58.420389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163702e-b8f7-448b-bab7-2d14f314189f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:18:58.438363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 567/1129] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:00.648932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22a9d1-0695-43ec-baad-2a8b867a1b8f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:00.649510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22a9d1-0695-43ec-baad-2a8b867a1b8f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:00.649510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22a9d1-0695-43ec-baad-2a8b867a1b8f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:00.650854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 563/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:00.657078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80787295-77ef-4d53-a37d-30b90637110e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:00.657532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80787295-77ef-4d53-a37d-30b90637110e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:00.657640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80787295-77ef-4d53-a37d-30b90637110e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:00.721089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:00.724216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-80787295-77ef-4d53-a37d-30b90637110e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:00.725994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-80787295-77ef-4d53-a37d-30b90637110e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:00.822812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 568/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:01.111008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22a9d1-0695-43ec-baad-2a8b867a1b8f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:01.112564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0d22a9d1-0695-43ec-baad-2a8b867a1b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:01.215969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22a9d1-0695-43ec-baad-2a8b867a1b8f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:01.265864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:01.289602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a88a4-cc47-4fbf-9749-632a95cafc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:01.290187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a88a4-cc47-4fbf-9749-632a95cafc54 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:01.290490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a88a4-cc47-4fbf-9749-632a95cafc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:01.290757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a88a4-cc47-4fbf-9749-632a95cafc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:01.292346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7b0a88a4-cc47-4fbf-9749-632a95cafc54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:01.399160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a88a4-cc47-4fbf-9749-632a95cafc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:01.400475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a88a4-cc47-4fbf-9749-632a95cafc54 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:01.418730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 564/1132] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:03.598676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fb74f-da07-4894-adbe-5e5540026171 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:03.599512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fb74f-da07-4894-adbe-5e5540026171 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:03.599512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fb74f-da07-4894-adbe-5e5540026171 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:03.600809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 569/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:03.607030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829789f1-05a3-41b7-a7e5-6ee55f2e2071 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:03.607454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829789f1-05a3-41b7-a7e5-6ee55f2e2071 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:03.607565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829789f1-05a3-41b7-a7e5-6ee55f2e2071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:03.672745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-829789f1-05a3-41b7-a7e5-6ee55f2e2071 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:03.673928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-829789f1-05a3-41b7-a7e5-6ee55f2e2071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:03.703357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:03.771881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 565/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:04.057440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fb74f-da07-4894-adbe-5e5540026171 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:04.059192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bf6fb74f-da07-4894-adbe-5e5540026171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:04.162829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fb74f-da07-4894-adbe-5e5540026171 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:04.192041 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:04.239894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0272a2-b4bc-4979-8a3f-f78042c74d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:04.240502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0272a2-b4bc-4979-8a3f-f78042c74d07 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:04.240809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0272a2-b4bc-4979-8a3f-f78042c74d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:04.241078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0272a2-b4bc-4979-8a3f-f78042c74d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:04.242716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3f0272a2-b4bc-4979-8a3f-f78042c74d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:04.351522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0272a2-b4bc-4979-8a3f-f78042c74d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:04.352881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0272a2-b4bc-4979-8a3f-f78042c74d07 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:04.371066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 570/1135] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:06.596610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9aff9-fda7-4a23-a043-9757e9f48986 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:06.597184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9aff9-fda7-4a23-a043-9757e9f48986 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:06.597184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9aff9-fda7-4a23-a043-9757e9f48986 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:06.598555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 566/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:06.604186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabca45-0216-43b0-8f5c-a2554816d84f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:06.605143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabca45-0216-43b0-8f5c-a2554816d84f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:06.605143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabca45-0216-43b0-8f5c-a2554816d84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:06.669731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5fabca45-0216-43b0-8f5c-a2554816d84f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:06.670934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5fabca45-0216-43b0-8f5c-a2554816d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:06.679094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:06.771402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 571/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:07.056256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9aff9-fda7-4a23-a043-9757e9f48986 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:07.058439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-05d9aff9-fda7-4a23-a043-9757e9f48986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:07.161214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9aff9-fda7-4a23-a043-9757e9f48986 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:07.216203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:07.235665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c17905-b5d5-41b3-bf95-e86f8c4674cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:07.236271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c17905-b5d5-41b3-bf95-e86f8c4674cb service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:07.236541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c17905-b5d5-41b3-bf95-e86f8c4674cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:07.236805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c17905-b5d5-41b3-bf95-e86f8c4674cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:07.238504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-52c17905-b5d5-41b3-bf95-e86f8c4674cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:07.351221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c17905-b5d5-41b3-bf95-e86f8c4674cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:07.352640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c17905-b5d5-41b3-bf95-e86f8c4674cb service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:07.371343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 567/1138] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:09.567323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dfb73-d7f6-462c-91e7-6c9ceda3e9dc None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:09.567842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dfb73-d7f6-462c-91e7-6c9ceda3e9dc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:09.567842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dfb73-d7f6-462c-91e7-6c9ceda3e9dc None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:09.569120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 572/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:09.574501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed293c-314b-4a78-b158-0dfa2d9f06c3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:09.574952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed293c-314b-4a78-b158-0dfa2d9f06c3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:09.575068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed293c-314b-4a78-b158-0dfa2d9f06c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:09.640543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-89ed293c-314b-4a78-b158-0dfa2d9f06c3 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:09.641737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-89ed293c-314b-4a78-b158-0dfa2d9f06c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:09.680032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:09.739137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 568/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:19:10.035344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dfb73-d7f6-462c-91e7-6c9ceda3e9dc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:10.036859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0d4dfb73-d7f6-462c-91e7-6c9ceda3e9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:10.141364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dfb73-d7f6-462c-91e7-6c9ceda3e9dc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:10.181933 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:10.218791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227439ab-d7e0-4327-bab0-d2cf64221df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:10.219391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227439ab-d7e0-4327-bab0-d2cf64221df5 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:10.219663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227439ab-d7e0-4327-bab0-d2cf64221df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:10.219927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227439ab-d7e0-4327-bab0-d2cf64221df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:10.221495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-227439ab-d7e0-4327-bab0-d2cf64221df5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:10.329931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227439ab-d7e0-4327-bab0-d2cf64221df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:10.331374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227439ab-d7e0-4327-bab0-d2cf64221df5 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:10.349275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 573/1141] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:12.553288 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73692-b630-4e39-9789-6d4cd2bec06d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:12.553855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73692-b630-4e39-9789-6d4cd2bec06d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:12.553855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73692-b630-4e39-9789-6d4cd2bec06d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:12.555184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 569/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:12.561086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d03fc9-9254-4a77-a655-a429fcb69b72 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:12.561515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d03fc9-9254-4a77-a655-a429fcb69b72 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:12.561610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d03fc9-9254-4a77-a655-a429fcb69b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:12.629561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-33d03fc9-9254-4a77-a655-a429fcb69b72 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:12.630834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-33d03fc9-9254-4a77-a655-a429fcb69b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:12.649572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:12.728202 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 574/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:13.019191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a73692-b630-4e39-9789-6d4cd2bec06d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:13.020708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d2a73692-b630-4e39-9789-6d4cd2bec06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:13.129417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a73692-b630-4e39-9789-6d4cd2bec06d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:13.196813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:13.204354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd1f3d-8cb5-4039-8534-caceab8b96f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:13.204951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd1f3d-8cb5-4039-8534-caceab8b96f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:13.205227 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd1f3d-8cb5-4039-8534-caceab8b96f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:13.205494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd1f3d-8cb5-4039-8534-caceab8b96f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:13.207178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e0fd1f3d-8cb5-4039-8534-caceab8b96f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:13.321580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd1f3d-8cb5-4039-8534-caceab8b96f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:13.323674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd1f3d-8cb5-4039-8534-caceab8b96f5 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:13.348144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 570/1144] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:15.592239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146c27e-820d-43cf-ab6e-52f4b09626c5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:15.592822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146c27e-820d-43cf-ab6e-52f4b09626c5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:15.592822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146c27e-820d-43cf-ab6e-52f4b09626c5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:15.594196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 575/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:15.599775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b65029-3806-4c08-98cf-be43e8ea302e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:15.600225 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b65029-3806-4c08-98cf-be43e8ea302e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:15.600334 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b65029-3806-4c08-98cf-be43e8ea302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:15.665392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-71b65029-3806-4c08-98cf-be43e8ea302e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:15.667394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-71b65029-3806-4c08-98cf-be43e8ea302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:15.715683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:15.765424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 571/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:19:16.054913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146c27e-820d-43cf-ab6e-52f4b09626c5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:16.056441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f146c27e-820d-43cf-ab6e-52f4b09626c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:16.161451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146c27e-820d-43cf-ab6e-52f4b09626c5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:16.210040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:16.236473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038c21d-a8b3-4a6d-a4c8-71259b6b1590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:16.237084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038c21d-a8b3-4a6d-a4c8-71259b6b1590 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:16.237356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038c21d-a8b3-4a6d-a4c8-71259b6b1590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:16.237621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038c21d-a8b3-4a6d-a4c8-71259b6b1590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:16.239292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c038c21d-a8b3-4a6d-a4c8-71259b6b1590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:16.348869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038c21d-a8b3-4a6d-a4c8-71259b6b1590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:16.350552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038c21d-a8b3-4a6d-a4c8-71259b6b1590 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:16.368425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 576/1147] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:25.763620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=22 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:26.366269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=66 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:36.325596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd43c10-aebb-405d-a7f4-fcfb1d953853 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:36.326137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd43c10-aebb-405d-a7f4-fcfb1d953853 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:36.326137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd43c10-aebb-405d-a7f4-fcfb1d953853 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:36.327492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 572/1148] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:36.332938 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a70ed-d025-45f7-bec8-405b693c9717 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:36.333361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a70ed-d025-45f7-bec8-405b693c9717 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:36.333469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a70ed-d025-45f7-bec8-405b693c9717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:36.400755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1a8a70ed-d025-45f7-bec8-405b693c9717 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:36.401941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1a8a70ed-d025-45f7-bec8-405b693c9717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:36.496107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 577/1149] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:36.793921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0636f-2423-4990-91d1-9cbd77f7b616 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:36.794202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0636f-2423-4990-91d1-9cbd77f7b616 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:36.794516 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0636f-2423-4990-91d1-9cbd77f7b616 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:36.795835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 573/1150] 10.222.0.16 () {54 vars in 962 bytes} [Mon Oct 17 10:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:36.800742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0aa55-25c1-4e50-967e-071fce2f60ee None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:36.801029 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0aa55-25c1-4e50-967e-071fce2f60ee None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:36.801303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0aa55-25c1-4e50-967e-071fce2f60ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:36.878466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-48d0aa55-25c1-4e50-967e-071fce2f60ee None None] MFA Rules not processed for user `9ac934ed0122470cb489d6b324c664f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:36.879648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-48d0aa55-25c1-4e50-967e-071fce2f60ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:36.899572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:36.954524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 578/1151] 10.222.0.16 () {58 vars in 1059 bytes} [Mon Oct 17 10:19:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:36.962286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0636f-2423-4990-91d1-9cbd77f7b616 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:36.963816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bdc0636f-2423-4990-91d1-9cbd77f7b616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:37.066063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0636f-2423-4990-91d1-9cbd77f7b616 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:37.139432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b944c-f06b-475e-af93-951536acf03f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ac934ed0122470cb489d6b324c664f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:37.140024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b944c-f06b-475e-af93-951536acf03f service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:37.140298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b944c-f06b-475e-af93-951536acf03f service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:37.140565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b944c-f06b-475e-af93-951536acf03f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:37.142137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-746b944c-f06b-475e-af93-951536acf03f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:37.197740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:37.256730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b944c-f06b-475e-af93-951536acf03f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:37.258150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b944c-f06b-475e-af93-951536acf03f service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:37.276919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 574/1152] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:39.520829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83046-d508-4690-a3ad-9ab6b2b7960a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:39.521382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83046-d508-4690-a3ad-9ab6b2b7960a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:39.521382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83046-d508-4690-a3ad-9ab6b2b7960a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:39.522725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 579/1153] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:39.528430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5fd2-5cbc-46e2-a561-e69a7a94108e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:39.528908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5fd2-5cbc-46e2-a561-e69a7a94108e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:39.529013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5fd2-5cbc-46e2-a561-e69a7a94108e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:39.594099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-14ea5fd2-5cbc-46e2-a561-e69a7a94108e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:39.595331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-14ea5fd2-5cbc-46e2-a561-e69a7a94108e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:39.691028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 575/1154] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:39.968952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc83046-d508-4690-a3ad-9ab6b2b7960a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:39.970491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2cc83046-d508-4690-a3ad-9ab6b2b7960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:40.095772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc83046-d508-4690-a3ad-9ab6b2b7960a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:40.145716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:40.191825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d797c87-281f-48b0-a358-39c4e4665ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:40.192429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797c87-281f-48b0-a358-39c4e4665ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:40.192701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797c87-281f-48b0-a358-39c4e4665ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:40.192982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797c87-281f-48b0-a358-39c4e4665ba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:40.193589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d797c87-281f-48b0-a358-39c4e4665ba1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:40.194655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-2d797c87-281f-48b0-a358-39c4e4665ba1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:19:40.194655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:19:40.194655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:19:40.195815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d797c87-281f-48b0-a358-39c4e4665ba1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:40.216404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 580/1155] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Oct 17 10:19:39 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:19:42.373104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818305-b134-42ad-917b-569f91d190ce None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:42.373638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818305-b134-42ad-917b-569f91d190ce None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:42.373638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818305-b134-42ad-917b-569f91d190ce None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:42.374933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 576/1156] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:42.380207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b879-bddb-4069-b1b4-08496d1e7b0b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:42.380689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b879-bddb-4069-b1b4-08496d1e7b0b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:42.380789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b879-bddb-4069-b1b4-08496d1e7b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:42.448374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1095b879-bddb-4069-b1b4-08496d1e7b0b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:42.449568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1095b879-bddb-4069-b1b4-08496d1e7b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:42.546606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 581/1157] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:42.839242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a818305-b134-42ad-917b-569f91d190ce None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:42.840836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0a818305-b134-42ad-917b-569f91d190ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:42.850945 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:42.985953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a818305-b134-42ad-917b-569f91d190ce None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:43.065268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:43.101335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd5be9-aa0a-4fc0-9719-e2b53400c6f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:43.101929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5be9-aa0a-4fc0-9719-e2b53400c6f0 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:43.102200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5be9-aa0a-4fc0-9719-e2b53400c6f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:43.102504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5be9-aa0a-4fc0-9719-e2b53400c6f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:43.104062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-55bd5be9-aa0a-4fc0-9719-e2b53400c6f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:43.193591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd5be9-aa0a-4fc0-9719-e2b53400c6f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:43.195224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd5be9-aa0a-4fc0-9719-e2b53400c6f0 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:43.213433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 577/1158] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:45.531794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af3759-2200-4feb-8644-9681cf383ef4 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:45.532402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af3759-2200-4feb-8644-9681cf383ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:45.532402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af3759-2200-4feb-8644-9681cf383ef4 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:45.533743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 582/1159] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:45.539059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d914b5-0990-4085-bda5-010e17c3fde7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:45.539478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d914b5-0990-4085-bda5-010e17c3fde7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:45.539607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d914b5-0990-4085-bda5-010e17c3fde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:45.606507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c4d914b5-0990-4085-bda5-010e17c3fde7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:45.607696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c4d914b5-0990-4085-bda5-010e17c3fde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:45.653699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:45.703074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 578/1160] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:46.002640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af3759-2200-4feb-8644-9681cf383ef4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:46.004184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e6af3759-2200-4feb-8644-9681cf383ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:46.029197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:46.109627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af3759-2200-4feb-8644-9681cf383ef4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:46.183648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6be81-e3a9-4ba2-b5ff-377320dc7872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:46.184250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6be81-e3a9-4ba2-b5ff-377320dc7872 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:46.184523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6be81-e3a9-4ba2-b5ff-377320dc7872 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:46.184788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6be81-e3a9-4ba2-b5ff-377320dc7872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:46.186398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cee6be81-e3a9-4ba2-b5ff-377320dc7872 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:46.261358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:46.296239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6be81-e3a9-4ba2-b5ff-377320dc7872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:46.297676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6be81-e3a9-4ba2-b5ff-377320dc7872 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:46.316128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 583/1161] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:48.531486 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962596c-a9bc-494c-a752-ed190d8c548c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:48.532046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962596c-a9bc-494c-a752-ed190d8c548c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:48.532046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962596c-a9bc-494c-a752-ed190d8c548c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:48.533404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 579/1162] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:48.538789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138e967-a968-4768-9cc7-93d998b361a5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:48.539199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138e967-a968-4768-9cc7-93d998b361a5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:48.539321 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138e967-a968-4768-9cc7-93d998b361a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:48.606822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0138e967-a968-4768-9cc7-93d998b361a5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:48.608023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0138e967-a968-4768-9cc7-93d998b361a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:48.703132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 584/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:48.989509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962596c-a9bc-494c-a752-ed190d8c548c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:48.991090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b962596c-a9bc-494c-a752-ed190d8c548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:49.096074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962596c-a9bc-494c-a752-ed190d8c548c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:49.168347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:49.169974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50d740-3b3b-4f80-885d-3d495b8dbfa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:49.170626 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d740-3b3b-4f80-885d-3d495b8dbfa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:49.171085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d740-3b3b-4f80-885d-3d495b8dbfa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:49.171507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d740-3b3b-4f80-885d-3d495b8dbfa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:49.173299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7e50d740-3b3b-4f80-885d-3d495b8dbfa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:49.282919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50d740-3b3b-4f80-885d-3d495b8dbfa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:49.284342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50d740-3b3b-4f80-885d-3d495b8dbfa6 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:49.303011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 580/1164] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:51.623289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26678e66-cac8-4d51-a39c-1e2bfb9f92eb None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:51.623855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26678e66-cac8-4d51-a39c-1e2bfb9f92eb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:51.623855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26678e66-cac8-4d51-a39c-1e2bfb9f92eb None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:51.625172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 585/1165] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:51.630494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb8746-6375-45cf-8b46-4800aee1140e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:51.630878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb8746-6375-45cf-8b46-4800aee1140e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:51.631036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb8746-6375-45cf-8b46-4800aee1140e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:51.696530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8efb8746-6375-45cf-8b46-4800aee1140e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:51.698427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8efb8746-6375-45cf-8b46-4800aee1140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:51.751566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:51.797063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 581/1166] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:52.097304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26678e66-cac8-4d51-a39c-1e2bfb9f92eb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:52.098870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-26678e66-cac8-4d51-a39c-1e2bfb9f92eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:52.135604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:52.203856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26678e66-cac8-4d51-a39c-1e2bfb9f92eb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:52.279048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef34807-8673-4fa4-a3d2-c25dcdb6368c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:52.279650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef34807-8673-4fa4-a3d2-c25dcdb6368c service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:52.279919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef34807-8673-4fa4-a3d2-c25dcdb6368c service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:52.280184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef34807-8673-4fa4-a3d2-c25dcdb6368c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:52.281765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7ef34807-8673-4fa4-a3d2-c25dcdb6368c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:52.362979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:52.391313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef34807-8673-4fa4-a3d2-c25dcdb6368c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:52.392827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef34807-8673-4fa4-a3d2-c25dcdb6368c service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:52.411511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 586/1167] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:19:54.811812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb069-c950-4f1c-825b-43108e2babbf None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:54.812383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb069-c950-4f1c-825b-43108e2babbf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:54.812383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb069-c950-4f1c-825b-43108e2babbf None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:54.813680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 582/1168] 10.222.0.16 () {54 vars in 927 bytes} [Mon Oct 17 10:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:19:54.818893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d700a4a-4cb0-4a7a-8dc9-aee7786c6bdf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:54.819299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d700a4a-4cb0-4a7a-8dc9-aee7786c6bdf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:54.819471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d700a4a-4cb0-4a7a-8dc9-aee7786c6bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:54.885073 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9d700a4a-4cb0-4a7a-8dc9-aee7786c6bdf None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:19:54.886269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9d700a4a-4cb0-4a7a-8dc9-aee7786c6bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:54.983047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 587/1169] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Oct 17 10:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:19:55.268596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abb069-c950-4f1c-825b-43108e2babbf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:19:55.270185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c2abb069-c950-4f1c-825b-43108e2babbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:55.373876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abb069-c950-4f1c-825b-43108e2babbf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:19:55.445730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa313b-8713-4c30-b91a-6a98b6ab9360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:19:55.446409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa313b-8713-4c30-b91a-6a98b6ab9360 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:19:55.446642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa313b-8713-4c30-b91a-6a98b6ab9360 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:19:55.446910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa313b-8713-4c30-b91a-6a98b6ab9360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:19:55.448509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-96fa313b-8713-4c30-b91a-6a98b6ab9360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:19:55.456933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:19:55.557639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa313b-8713-4c30-b91a-6a98b6ab9360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:19:55.559083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa313b-8713-4c30-b91a-6a98b6ab9360 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:19:55.577635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 583/1170] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:20:04.981517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=82 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:20:05.575279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=50 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:08.411153 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbc0a8-25f6-41d2-b049-ad097185f144 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:08.411715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbc0a8-25f6-41d2-b049-ad097185f144 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:08.411715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbc0a8-25f6-41d2-b049-ad097185f144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:08.486545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-86fbc0a8-25f6-41d2-b049-ad097185f144 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:21:08.487736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-86fbc0a8-25f6-41d2-b049-ad097185f144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:08.586426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 588/1171] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:21:08.594442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea864a55-f762-4cc0-92ba-c29fb2fbd3e4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:08.594934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea864a55-f762-4cc0-92ba-c29fb2fbd3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:08.595040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea864a55-f762-4cc0-92ba-c29fb2fbd3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:08.669951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ea864a55-f762-4cc0-92ba-c29fb2fbd3e4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:21:08.671100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ea864a55-f762-4cc0-92ba-c29fb2fbd3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:08.765262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 584/1172] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:21:08.769893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbc0a8-25f6-41d2-b049-ad097185f144 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:08.771279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-86fbc0a8-25f6-41d2-b049-ad097185f144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:08.782932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbc0a8-25f6-41d2-b049-ad097185f144 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:08.844791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:08.878117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:08.878739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:08.879013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:08.879278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:08.879910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:21:08.880206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:08.881211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:21:08.881211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:21:08.881211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:21:08.882113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:08.893772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 589/1173] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:21:08.903412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea864a55-f762-4cc0-92ba-c29fb2fbd3e4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:08.912576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea864a55-f762-4cc0-92ba-c29fb2fbd3e4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:08.974561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:09.009331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:09.009921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:09.010196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:09.010491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:09.011126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:09.012217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:21:09.012217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Oct 17 10:21:09.012217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: Oct 17 10:21:09.013430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:09.071374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 585/1174] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:21:08 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:21:09.072556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:09.074072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:09.108428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafe38c-59ff-4e90-8e22-97eef78f3c7d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:09.202384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:09.202910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:09.203185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:09.203452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:09.204116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:09.205157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:21:09.205157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:21:09.205157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:21:09.206426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:09.249770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 590/1175] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:21:09 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:21:09.254244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:09.255808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:09.319298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915da21e-25e1-4ade-8ad4-c0b988541f53 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:09.403991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:09.415499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:09.416096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:09.416369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:09.416634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:09.417272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:09.418892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:09.431240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 586/1176] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:21:09.435268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:09.436816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:09.448572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272a694-3401-4e5f-902e-c136b3e875b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:09.473691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:09.543372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e6e9f-3cfe-424f-916a-cc2e49a41801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:09.543969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e6e9f-3cfe-424f-916a-cc2e49a41801 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:09.544241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e6e9f-3cfe-424f-916a-cc2e49a41801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:09.544507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e6e9f-3cfe-424f-916a-cc2e49a41801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:09.545144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e6e9f-3cfe-424f-916a-cc2e49a41801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:09.546761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e6e9f-3cfe-424f-916a-cc2e49a41801 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:09.559396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 591/1177] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:21:09.563506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:09.565067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:09.599016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440869c4-bba4-45f8-8f2a-e96ae25117a6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:09.691463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded2c9a-f034-4121-8e6f-f23e2542a1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:09.692105 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded2c9a-f034-4121-8e6f-f23e2542a1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:09.692381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded2c9a-f034-4121-8e6f-f23e2542a1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:09.692652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded2c9a-f034-4121-8e6f-f23e2542a1dd admin admin] PATH_INFO: `/v3/projects/e84f68db36cc4eb0a7c09f005bc87ace/users/201accb55bca402c9c7388973326a448/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:09.715652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded2c9a-f034-4121-8e6f-f23e2542a1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84f68db36cc4eb0a7c09f005bc87ace, user_id=201accb55bca402c9c7388973326a448, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:09.718289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded2c9a-f034-4121-8e6f-f23e2542a1dd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:09.742056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 587/1178] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:21:09 2022] PUT /identity/v3/projects/e84f68db36cc4eb0a7c09f005bc87ace/users/201accb55bca402c9c7388973326a448/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:21:09.753779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a5d4-8559-4106-a4af-6dba15e222ed None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:09.754071 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a5d4-8559-4106-a4af-6dba15e222ed None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:09.754396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a5d4-8559-4106-a4af-6dba15e222ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:09.812143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4291a5d4-8559-4106-a4af-6dba15e222ed None None] MFA Rules not processed for user `201accb55bca402c9c7388973326a448`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:21:09.813331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4291a5d4-8559-4106-a4af-6dba15e222ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:09.891551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 592/1179] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:21:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:21:09.905073 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693606-1f63-44cf-890a-f61b4af61b86 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:09.905358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693606-1f63-44cf-890a-f61b4af61b86 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:09.905631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693606-1f63-44cf-890a-f61b4af61b86 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:09.907012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 588/1180] 10.222.0.16 () {54 vars in 963 bytes} [Mon Oct 17 10:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:21:09.911849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352725-f403-43b7-a6d0-7b64547e1e6e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:09.912134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352725-f403-43b7-a6d0-7b64547e1e6e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:09.912390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352725-f403-43b7-a6d0-7b64547e1e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:09.944186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:09.989645 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e9352725-f403-43b7-a6d0-7b64547e1e6e None None] MFA Rules not processed for user `24f1a6c05ed94f5fa81dd355497b6754`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:21:09.990847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e9352725-f403-43b7-a6d0-7b64547e1e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:10.066882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 593/1181] 10.222.0.16 () {58 vars in 1060 bytes} [Mon Oct 17 10:21:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:21:10.075728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f693606-1f63-44cf-890a-f61b4af61b86 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:10.077235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f693606-1f63-44cf-890a-f61b4af61b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:10.127585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:10.183099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f693606-1f63-44cf-890a-f61b4af61b86 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:10.261605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:10.262205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:10.262512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:10.262780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:10.264423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:10.363636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:10.364958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:10.368553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:10.383119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 589/1182] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:21:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:21:10.428638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc34900-a59d-4714-9523-c11bebb1a34c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:10.428926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc34900-a59d-4714-9523-c11bebb1a34c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:10.429208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc34900-a59d-4714-9523-c11bebb1a34c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:10.430638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 594/1183] 10.222.0.16 () {52 vars in 830 bytes} [Mon Oct 17 10:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:21:10.434348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:10.435584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:10.535477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6aec9-f884-4da8-9ee5-fff07b6969df service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:10.633871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:10.634816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:10.635084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:10.635344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] PATH_INFO: `/v3/users/201accb55bca402c9c7388973326a448` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:10.648514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] RBAC: Authorizing `identity:delete_user(user_id=201accb55bca402c9c7388973326a448)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:10.649724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:21:10.649724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Oct 17 10:21:10.649724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: Oct 17 10:21:10.651153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:10.660913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:10.717536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201accb55bca402c9c7388973326a448'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:10.717967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201accb55bca402c9c7388973326a448'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:10.726635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201accb55bca402c9c7388973326a448'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:10.750165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201accb55bca402c9c7388973326a448'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:10.750727 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201accb55bca402c9c7388973326a448'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:10.770063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 590/1184] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:21:10 2022] DELETE /identity/v3/users/201accb55bca402c9c7388973326a448 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:21:10.774726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84599-a074-4baf-a59b-253dffd08979 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:10.775259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84599-a074-4baf-a59b-253dffd08979 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:10.775380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84599-a074-4baf-a59b-253dffd08979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:10.831727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f3a84599-a074-4baf-a59b-253dffd08979 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:21:10.832900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f3a84599-a074-4baf-a59b-253dffd08979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:10.907579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:10.921750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 595/1185] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:21:10.932137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:10.933352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:11.033276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4171c-0689-4876-a5fd-ba390f79707c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:11.089512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:11.111149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f0cc8-d3d2-423f-bd41-21ca325a73c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:11.111750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0cc8-d3d2-423f-bd41-21ca325a73c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:11.112379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0cc8-d3d2-423f-bd41-21ca325a73c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:11.112659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0cc8-d3d2-423f-bd41-21ca325a73c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:11.114259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb0f0cc8-d3d2-423f-bd41-21ca325a73c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:11.226429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f0cc8-d3d2-423f-bd41-21ca325a73c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:11.228574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f0cc8-d3d2-423f-bd41-21ca325a73c7 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:11.247743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 591/1186] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:21:11.573587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a84599-a074-4baf-a59b-253dffd08979 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:21:11.575038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f3a84599-a074-4baf-a59b-253dffd08979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:21:11.586998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a84599-a074-4baf-a59b-253dffd08979 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:21:11.688608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:21:11.689231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:21:11.689501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:21:11.689764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] PATH_INFO: `/v3/projects/e84f68db36cc4eb0a7c09f005bc87ace` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:21:11.700603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84f68db36cc4eb0a7c09f005bc87ace)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:21:11.701654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:21:11.701654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:21:11.701654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:21:11.703097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:21:11.762712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84f68db36cc4eb0a7c09f005bc87ace', 'request_id': 'req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:11.776698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84f68db36cc4eb0a7c09f005bc87ace', 'request_id': 'req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:11.786312 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84f68db36cc4eb0a7c09f005bc87ace', 'request_id': 'req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:11.809465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] The token cache is being invalidate because project e84f68db36cc4eb0a7c09f005bc87ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:21:11.809747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:11.810256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dba1b50a-b4dd-4ce3-a8ac-774f7c8a6ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:21:11.811694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 596/1187] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:21:11 2022] DELETE /identity/v3/projects/e84f68db36cc4eb0a7c09f005bc87ace => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:21:21.245531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=62 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:21:21.807517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:22:38.779181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888db3-0b57-44c3-b2b3-1bf5ef68a84d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:38.779795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888db3-0b57-44c3-b2b3-1bf5ef68a84d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:38.779921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888db3-0b57-44c3-b2b3-1bf5ef68a84d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:38.781479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 592/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:22:38.787407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735a2b-d27a-435b-aa3e-d0467a99aab9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:38.787883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735a2b-d27a-435b-aa3e-d0467a99aab9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:38.787998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735a2b-d27a-435b-aa3e-d0467a99aab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:38.871453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bd735a2b-d27a-435b-aa3e-d0467a99aab9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:22:38.872648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bd735a2b-d27a-435b-aa3e-d0467a99aab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:38.976561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 597/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:22:39.133244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93888db3-0b57-44c3-b2b3-1bf5ef68a84d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:22:39.135525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-93888db3-0b57-44c3-b2b3-1bf5ef68a84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:39.253284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93888db3-0b57-44c3-b2b3-1bf5ef68a84d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:22:39.258726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fe255-0ace-49a4-b726-562f767d74da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:22:39.259320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fe255-0ace-49a4-b726-562f767d74da service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:39.259594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fe255-0ace-49a4-b726-562f767d74da service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:39.259864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fe255-0ace-49a4-b726-562f767d74da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:39.267804 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fe255-0ace-49a4-b726-562f767d74da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:22:39.269373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fe255-0ace-49a4-b726-562f767d74da service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:22:39.279182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 593/1190] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:22:40.372295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcd8a8-8c21-469c-8691-b2c9db8c35fc None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:40.372829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcd8a8-8c21-469c-8691-b2c9db8c35fc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:40.372829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcd8a8-8c21-469c-8691-b2c9db8c35fc None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:40.374083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 598/1191] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:22:40.380302 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432575-13e9-451d-8f42-014d21d4f59c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:40.380739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432575-13e9-451d-8f42-014d21d4f59c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:40.380839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432575-13e9-451d-8f42-014d21d4f59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:40.401420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3d432575-13e9-451d-8f42-014d21d4f59c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:22:40.402657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3d432575-13e9-451d-8f42-014d21d4f59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:40.410466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 594/1192] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:22:40.562555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcd8a8-8c21-469c-8691-b2c9db8c35fc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:22:40.577858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcd8a8-8c21-469c-8691-b2c9db8c35fc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:22:40.583658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d926f-3f16-4271-8937-2b60819ce706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:22:40.584260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d926f-3f16-4271-8937-2b60819ce706 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:40.584538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d926f-3f16-4271-8937-2b60819ce706 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:40.584808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d926f-3f16-4271-8937-2b60819ce706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:40.593984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d926f-3f16-4271-8937-2b60819ce706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:22:40.595663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d926f-3f16-4271-8937-2b60819ce706 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:22:40.607320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 599/1193] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:22:41.633209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c0d4-4dfa-4cb2-8b22-5d14e4749441 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:41.633910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c0d4-4dfa-4cb2-8b22-5d14e4749441 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:41.634189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c0d4-4dfa-4cb2-8b22-5d14e4749441 None None] PATH_INFO: `/v3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:41.635490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 595/1194] 10.222.0.30 () {54 vars in 925 bytes} [Mon Oct 17 10:22:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:22:41.643310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2cdcd-eaeb-46b9-bb88-5ff6f9399898 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:41.643740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2cdcd-eaeb-46b9-bb88-5ff6f9399898 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:41.643848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2cdcd-eaeb-46b9-bb88-5ff6f9399898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:41.703679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-93e2cdcd-eaeb-46b9-bb88-5ff6f9399898 None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:22:41.704863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-93e2cdcd-eaeb-46b9-bb88-5ff6f9399898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:41.721377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:22:41.777241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 600/1195] 10.222.0.30 () {58 vars in 1015 bytes} [Mon Oct 17 10:22:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:22:50.395728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=56 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:22:50.613771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b91b0-f118-4e28-b3f2-9ad1070691c8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:50.614053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b91b0-f118-4e28-b3f2-9ad1070691c8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:50.614345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b91b0-f118-4e28-b3f2-9ad1070691c8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:50.615699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 596/1196] 10.222.0.16 () {54 vars in 961 bytes} [Mon Oct 17 10:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:22:50.621144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51b6bd-64e0-4d03-bad7-a32be8e2d512 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:50.621631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51b6bd-64e0-4d03-bad7-a32be8e2d512 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:50.621731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51b6bd-64e0-4d03-bad7-a32be8e2d512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:50.642658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-de51b6bd-64e0-4d03-bad7-a32be8e2d512 None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:22:50.643855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-de51b6bd-64e0-4d03-bad7-a32be8e2d512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:50.652415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 601/1197] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Oct 17 10:22:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:22:50.659873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b91b0-f118-4e28-b3f2-9ad1070691c8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:22:50.673695 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b91b0-f118-4e28-b3f2-9ad1070691c8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:22:50.678584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb7e5-23c9-450d-aa67-1d78272589b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:22:50.679164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb7e5-23c9-450d-aa67-1d78272589b2 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:50.679439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb7e5-23c9-450d-aa67-1d78272589b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:50.679706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb7e5-23c9-450d-aa67-1d78272589b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:50.688497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb7e5-23c9-450d-aa67-1d78272589b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:22:50.690068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb7e5-23c9-450d-aa67-1d78272589b2 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:22:50.699575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 597/1198] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:22:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:22:52.060795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e689655-05fe-4df6-ae55-d1bb5d57be90 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:52.061346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e689655-05fe-4df6-ae55-d1bb5d57be90 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:52.061346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e689655-05fe-4df6-ae55-d1bb5d57be90 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:52.062689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 602/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:22:52.068728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06b6ca-7933-45b7-9f62-ee3b1676157f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:52.069209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06b6ca-7933-45b7-9f62-ee3b1676157f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:52.069308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06b6ca-7933-45b7-9f62-ee3b1676157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:52.089216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cf06b6ca-7933-45b7-9f62-ee3b1676157f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:22:52.090429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cf06b6ca-7933-45b7-9f62-ee3b1676157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:52.098148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 598/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:22:52.256921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e689655-05fe-4df6-ae55-d1bb5d57be90 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:22:52.271272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e689655-05fe-4df6-ae55-d1bb5d57be90 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:22:52.277305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066c797-b201-4d71-a201-2da907fdf23a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:22:52.277891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c797-b201-4d71-a201-2da907fdf23a service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:52.278163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c797-b201-4d71-a201-2da907fdf23a service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:52.278461 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c797-b201-4d71-a201-2da907fdf23a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:52.286231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066c797-b201-4d71-a201-2da907fdf23a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:22:52.287842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066c797-b201-4d71-a201-2da907fdf23a service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:22:52.296909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 603/1201] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:22:53.361479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f735-2a0f-4902-a7a2-c9e13942309f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:53.362050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f735-2a0f-4902-a7a2-c9e13942309f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:53.362050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f735-2a0f-4902-a7a2-c9e13942309f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:53.363400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 599/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 10:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:22:53.369517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25181a3d-3c03-45e1-b96e-9702f8bfa347 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:53.369984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25181a3d-3c03-45e1-b96e-9702f8bfa347 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:53.370115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25181a3d-3c03-45e1-b96e-9702f8bfa347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:53.389019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-25181a3d-3c03-45e1-b96e-9702f8bfa347 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:22:53.390201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-25181a3d-3c03-45e1-b96e-9702f8bfa347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:53.397888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 604/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:22:53.549169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0f735-2a0f-4902-a7a2-c9e13942309f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:22:53.563646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0f735-2a0f-4902-a7a2-c9e13942309f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:22:53.568896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ed3a8-8a36-4e51-a3ff-7348044b11e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:22:53.569471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed3a8-8a36-4e51-a3ff-7348044b11e0 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:53.569742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed3a8-8a36-4e51-a3ff-7348044b11e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:53.570006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed3a8-8a36-4e51-a3ff-7348044b11e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:53.579826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ed3a8-8a36-4e51-a3ff-7348044b11e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:22:53.581370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ed3a8-8a36-4e51-a3ff-7348044b11e0 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:22:53.591633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 600/1204] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:22:57.612489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c70c33-1fae-4ae8-aa84-1d607fee73b3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:57.613070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c70c33-1fae-4ae8-aa84-1d607fee73b3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:57.613070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c70c33-1fae-4ae8-aa84-1d607fee73b3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:57.614396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 605/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:22:57.620588 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003b8e-428b-4430-912e-0284773d45d4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:57.621058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003b8e-428b-4430-912e-0284773d45d4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:57.621169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003b8e-428b-4430-912e-0284773d45d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:57.640330 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9c003b8e-428b-4430-912e-0284773d45d4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:22:57.641493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9c003b8e-428b-4430-912e-0284773d45d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:57.648988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 601/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:22:57.947089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c70c33-1fae-4ae8-aa84-1d607fee73b3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:22:57.948804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-41c70c33-1fae-4ae8-aa84-1d607fee73b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:22:58.027205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c70c33-1fae-4ae8-aa84-1d607fee73b3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:22:58.030714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4f176-39b3-45a0-b481-996d171bd2d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:22:58.031307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4f176-39b3-45a0-b481-996d171bd2d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:22:58.031581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4f176-39b3-45a0-b481-996d171bd2d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:22:58.031851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4f176-39b3-45a0-b481-996d171bd2d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:22:58.040605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4f176-39b3-45a0-b481-996d171bd2d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:22:58.042022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4f176-39b3-45a0-b481-996d171bd2d6 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:22:58.051150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 606/1207] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:23:00.244850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583845b0-ca55-43ed-a8d0-57185046dbe9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:00.245428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583845b0-ca55-43ed-a8d0-57185046dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:00.245428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583845b0-ca55-43ed-a8d0-57185046dbe9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:00.246817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 602/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:23:00.252229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2645-cbd8-497d-aab0-426cacf6eed0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:00.252682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2645-cbd8-497d-aab0-426cacf6eed0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:00.252776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2645-cbd8-497d-aab0-426cacf6eed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:00.272417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b0fd2645-cbd8-497d-aab0-426cacf6eed0 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:23:00.273597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b0fd2645-cbd8-497d-aab0-426cacf6eed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:23:00.281403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 607/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:23:00.569806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583845b0-ca55-43ed-a8d0-57185046dbe9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:23:00.580517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583845b0-ca55-43ed-a8d0-57185046dbe9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:23:00.583996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5a856-180e-4e91-9b14-a5798003638f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:23:00.584580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5a856-180e-4e91-9b14-a5798003638f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:00.584854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5a856-180e-4e91-9b14-a5798003638f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:00.585122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5a856-180e-4e91-9b14-a5798003638f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:00.593870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5a856-180e-4e91-9b14-a5798003638f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:23:00.595310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5a856-180e-4e91-9b14-a5798003638f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:23:00.605527 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 603/1210] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:23:04.129043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bcd3d-7b78-4eff-8740-852a0090f7ce None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:04.129574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bcd3d-7b78-4eff-8740-852a0090f7ce None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:04.129574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bcd3d-7b78-4eff-8740-852a0090f7ce None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:04.130972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 608/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:23:04.136316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67736507-1dd7-4f19-8b02-382ac2e84405 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:04.136758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67736507-1dd7-4f19-8b02-382ac2e84405 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:04.136856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67736507-1dd7-4f19-8b02-382ac2e84405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:04.156168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-67736507-1dd7-4f19-8b02-382ac2e84405 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:23:04.157350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67736507-1dd7-4f19-8b02-382ac2e84405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:23:04.165820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 604/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:23:04.468264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bcd3d-7b78-4eff-8740-852a0090f7ce None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:23:04.469884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f13bcd3d-7b78-4eff-8740-852a0090f7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:23:04.535155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:23:04.549385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bcd3d-7b78-4eff-8740-852a0090f7ce None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:23:04.552859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea713434-c0e3-45ba-8e4f-05e1a68cfeae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:23:04.553451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713434-c0e3-45ba-8e4f-05e1a68cfeae service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:04.553724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713434-c0e3-45ba-8e4f-05e1a68cfeae service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:04.553989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713434-c0e3-45ba-8e4f-05e1a68cfeae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:04.563244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea713434-c0e3-45ba-8e4f-05e1a68cfeae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:23:04.564698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea713434-c0e3-45ba-8e4f-05e1a68cfeae service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:23:04.574315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 609/1213] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:23:06.735593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ab87-0d7a-490b-9eb0-60b9ed5a0d41 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:06.736155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ab87-0d7a-490b-9eb0-60b9ed5a0d41 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:06.736155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ab87-0d7a-490b-9eb0-60b9ed5a0d41 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:06.737469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 605/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 10:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:23:06.743553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfba71-3e86-45a2-b4d3-1f71299eaa47 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:06.744018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfba71-3e86-45a2-b4d3-1f71299eaa47 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:06.744297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfba71-3e86-45a2-b4d3-1f71299eaa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:06.763986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-82bfba71-3e86-45a2-b4d3-1f71299eaa47 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:23:06.765166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-82bfba71-3e86-45a2-b4d3-1f71299eaa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:23:06.772870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 610/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 10:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:23:07.079009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897ab87-0d7a-490b-9eb0-60b9ed5a0d41 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:23:07.080671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0897ab87-0d7a-490b-9eb0-60b9ed5a0d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:23:07.094234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897ab87-0d7a-490b-9eb0-60b9ed5a0d41 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:23:07.097683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78580c-64a2-4984-b162-9ded05b35efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:23:07.098265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78580c-64a2-4984-b162-9ded05b35efe service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:23:07.098567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78580c-64a2-4984-b162-9ded05b35efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:23:07.098835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78580c-64a2-4984-b162-9ded05b35efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:23:07.107790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78580c-64a2-4984-b162-9ded05b35efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:23:07.109184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78580c-64a2-4984-b162-9ded05b35efe service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:23:07.118269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 606/1216] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:23:16.759075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=11 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:23:17.114234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=33 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:31.435657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973867-3fe4-4c3e-a5be-a35d6d2753e4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.436805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973867-3fe4-4c3e-a5be-a35d6d2753e4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.437307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973867-3fe4-4c3e-a5be-a35d6d2753e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.465884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1f973867-3fe4-4c3e-a5be-a35d6d2753e4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.467521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1f973867-3fe4-4c3e-a5be-a35d6d2753e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.479339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 611/1217] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:31.490456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d4cbc-1a49-4cb9-9ea2-98dd9b25077d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.491067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d4cbc-1a49-4cb9-9ea2-98dd9b25077d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.491067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d4cbc-1a49-4cb9-9ea2-98dd9b25077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.527178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-904d4cbc-1a49-4cb9-9ea2-98dd9b25077d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.528383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-904d4cbc-1a49-4cb9-9ea2-98dd9b25077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.541589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 607/1218] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:31.546764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f973867-3fe4-4c3e-a5be-a35d6d2753e4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:31.558296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f973867-3fe4-4c3e-a5be-a35d6d2753e4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:31.568981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:31.570017 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.570568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.571187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.571317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989799a-37f4-4165-aaae-3eba798c21c2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.571599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989799a-37f4-4165-aaae-3eba798c21c2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.571878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989799a-37f4-4165-aaae-3eba798c21c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.572680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:31.573233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:31.575171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:31.599440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 612/1219] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:31.604810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6989799a-37f4-4165-aaae-3eba798c21c2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.606026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6989799a-37f4-4165-aaae-3eba798c21c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.609081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:31.621822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 608/1220] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:24:31.625200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f266ba5-feb4-44d2-b75a-9b0987153ac8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:31.637012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be30d-afe8-4cb1-bc77-0e64c5ebc21f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.637372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be30d-afe8-4cb1-bc77-0e64c5ebc21f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.637569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be30d-afe8-4cb1-bc77-0e64c5ebc21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.639596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:31.640501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.641244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.641742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.642623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:31.644144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:24:31.644144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:24:31.644144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:24:31.645781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:31.668352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2d4be30d-afe8-4cb1-bc77-0e64c5ebc21f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.669567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2d4be30d-afe8-4cb1-bc77-0e64c5ebc21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.689203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 609/1221] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:24:31.690582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e92d3c-08cb-4f09-83c0-d0acceb8b9ab None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.690859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e92d3c-08cb-4f09-83c0-d0acceb8b9ab None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.691299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e92d3c-08cb-4f09-83c0-d0acceb8b9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.715652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 613/1222] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:31.717675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:31.733306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-28e92d3c-08cb-4f09-83c0-d0acceb8b9ab None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.735022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-28e92d3c-08cb-4f09-83c0-d0acceb8b9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.808845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5331870-8155-4b78-bf15-0bea13f22e73 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:31.814467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f30030-2b7c-4acd-9e7d-9662de2ce129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:31.815335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f30030-2b7c-4acd-9e7d-9662de2ce129 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.815831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f30030-2b7c-4acd-9e7d-9662de2ce129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.816332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f30030-2b7c-4acd-9e7d-9662de2ce129 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.816985 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f30030-2b7c-4acd-9e7d-9662de2ce129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:31.817287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f30030-2b7c-4acd-9e7d-9662de2ce129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:31.819110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f30030-2b7c-4acd-9e7d-9662de2ce129 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:31.823748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 610/1223] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:31.826193 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e92d3c-08cb-4f09-83c0-d0acceb8b9ab None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:31.832790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 614/1224] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:31.834080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a708b-3704-46d9-a761-8016c9d3a0b6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.837792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a708b-3704-46d9-a761-8016c9d3a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.837792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a708b-3704-46d9-a761-8016c9d3a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.843882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e92d3c-08cb-4f09-83c0-d0acceb8b9ab None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:31.851382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f085644-fa26-4357-b84c-22903a9e128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:31.853190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085644-fa26-4357-b84c-22903a9e128f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.853808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085644-fa26-4357-b84c-22903a9e128f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.854445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085644-fa26-4357-b84c-22903a9e128f admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.855655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f085644-fa26-4357-b84c-22903a9e128f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:31.857472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-5f085644-fa26-4357-b84c-22903a9e128f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:24:31.857472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Oct 17 10:24:31.857472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: Oct 17 10:24:31.859966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f085644-fa26-4357-b84c-22903a9e128f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:31.871402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-383a708b-3704-46d9-a761-8016c9d3a0b6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.873268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-383a708b-3704-46d9-a761-8016c9d3a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.891179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 615/1225] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:31.891179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e5173-59d0-44a7-bace-673115e212d6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.891179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e5173-59d0-44a7-bace-673115e212d6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.891179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e5173-59d0-44a7-bace-673115e212d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.902445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 611/1226] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/users => generated 290 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:31.904657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32874a84-397c-4c75-8929-0ba9d02fc099 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.905135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32874a84-397c-4c75-8929-0ba9d02fc099 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.905472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32874a84-397c-4c75-8929-0ba9d02fc099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.923198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1e0e5173-59d0-44a7-bace-673115e212d6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.925088 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1e0e5173-59d0-44a7-bace-673115e212d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.934426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-32874a84-397c-4c75-8929-0ba9d02fc099 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.936951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-32874a84-397c-4c75-8929-0ba9d02fc099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.938075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 616/1227] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:31.940102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7b75d-3e3e-4eed-b72a-2d7884b512a6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.940500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7b75d-3e3e-4eed-b72a-2d7884b512a6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.940949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7b75d-3e3e-4eed-b72a-2d7884b512a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.950216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 612/1228] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:31.952056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366d182-b6ed-4647-98ad-5f587ad8a49c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.952525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366d182-b6ed-4647-98ad-5f587ad8a49c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.953204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366d182-b6ed-4647-98ad-5f587ad8a49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.970564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7eb7b75d-3e3e-4eed-b72a-2d7884b512a6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.972407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7eb7b75d-3e3e-4eed-b72a-2d7884b512a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.982281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 617/1229] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:31.982575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f366d182-b6ed-4647-98ad-5f587ad8a49c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:31.983059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7b75d-3e3e-4eed-b72a-2d7884b512a6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:31.983757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f366d182-b6ed-4647-98ad-5f587ad8a49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:31.992383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 613/1230] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:31.993741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7b75d-3e3e-4eed-b72a-2d7884b512a6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:31.994472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e58ca-8db7-418f-87ae-8ef8b3bfcb73 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.994761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e58ca-8db7-418f-87ae-8ef8b3bfcb73 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.995027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e58ca-8db7-418f-87ae-8ef8b3bfcb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:31.998594 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bdb5d-74e3-4237-8398-a2eeddf48fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:31.999221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bdb5d-74e3-4237-8398-a2eeddf48fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:31.999496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bdb5d-74e3-4237-8398-a2eeddf48fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:31.999760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bdb5d-74e3-4237-8398-a2eeddf48fe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.000383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bdb5d-74e3-4237-8398-a2eeddf48fe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.002278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bdb5d-74e3-4237-8398-a2eeddf48fe2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.014444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1c3e58ca-8db7-418f-87ae-8ef8b3bfcb73 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:32.015678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1c3e58ca-8db7-418f-87ae-8ef8b3bfcb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:32.025463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 614/1231] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:32.026912 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e36e1-411e-48ca-b087-31d8de5e517a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.027189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e36e1-411e-48ca-b087-31d8de5e517a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.027456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e36e1-411e-48ca-b087-31d8de5e517a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.032455 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 618/1232] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:31 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.033380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bdb5d-74e3-4237-8398-a2eeddf48fe2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.048484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-067e36e1-411e-48ca-b087-31d8de5e517a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:32.049668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-067e36e1-411e-48ca-b087-31d8de5e517a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:32.102977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bdb5d-74e3-4237-8398-a2eeddf48fe2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.107718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d594d1f-e990-4521-8258-7efc53b7bd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.108328 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d594d1f-e990-4521-8258-7efc53b7bd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.108659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d594d1f-e990-4521-8258-7efc53b7bd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.108942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d594d1f-e990-4521-8258-7efc53b7bd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.109585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d594d1f-e990-4521-8258-7efc53b7bd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.110521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 615/1233] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:32.119197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d594d1f-e990-4521-8258-7efc53b7bd04 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.119329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965eb7e1-9458-442f-af10-6c4082d2f9f3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.119329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965eb7e1-9458-442f-af10-6c4082d2f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.119329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965eb7e1-9458-442f-af10-6c4082d2f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.119598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:32.127069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 619/1234] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:32.128257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23133a0d-4298-4d6c-861d-23a0e2408407 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.128544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23133a0d-4298-4d6c-861d-23a0e2408407 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.128821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23133a0d-4298-4d6c-861d-23a0e2408407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.136379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-965eb7e1-9458-442f-af10-6c4082d2f9f3 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:32.137569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-965eb7e1-9458-442f-af10-6c4082d2f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:32.145336 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 616/1235] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:32.146621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244635e2-a1c9-450e-8eb7-51ea155b5ef9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.146901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244635e2-a1c9-450e-8eb7-51ea155b5ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.147168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244635e2-a1c9-450e-8eb7-51ea155b5ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.150661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-23133a0d-4298-4d6c-861d-23a0e2408407 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:32.151848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-23133a0d-4298-4d6c-861d-23a0e2408407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:32.159804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 620/1236] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:32.163428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23133a0d-4298-4d6c-861d-23a0e2408407 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.167256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-244635e2-a1c9-450e-8eb7-51ea155b5ef9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:32.168427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-244635e2-a1c9-450e-8eb7-51ea155b5ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:32.172668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23133a0d-4298-4d6c-861d-23a0e2408407 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.176238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 617/1237] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:32.177002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244635e2-a1c9-450e-8eb7-51ea155b5ef9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.179436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.180105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.180982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.181417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.182153 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:32.182577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.184189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.186980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244635e2-a1c9-450e-8eb7-51ea155b5ef9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.191545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ea024-87e0-4912-b681-d0ae3cb6eac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.192058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ea024-87e0-4912-b681-d0ae3cb6eac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.192330 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ea024-87e0-4912-b681-d0ae3cb6eac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.192596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ea024-87e0-4912-b681-d0ae3cb6eac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.193243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ea024-87e0-4912-b681-d0ae3cb6eac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:32.193537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ea024-87e0-4912-b681-d0ae3cb6eac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.194561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-097ea024-87e0-4912-b681-d0ae3cb6eac2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:24:32.194561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Oct 17 10:24:32.194561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: Oct 17 10:24:32.195433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ea024-87e0-4912-b681-d0ae3cb6eac2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.196035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 621/1238] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:32.197119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.206686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6727089-094f-4a46-a139-b6a91658b5e4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.214511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 618/1239] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:32.214511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6496e-a288-430c-aef2-a7ee97443ddd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.214511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6496e-a288-430c-aef2-a7ee97443ddd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.214511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6496e-a288-430c-aef2-a7ee97443ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.217816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a091fe-45e0-4fb5-bb10-8d173de0a80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.218560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a091fe-45e0-4fb5-bb10-8d173de0a80c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.218866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a091fe-45e0-4fb5-bb10-8d173de0a80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.219132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a091fe-45e0-4fb5-bb10-8d173de0a80c admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.219747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a091fe-45e0-4fb5-bb10-8d173de0a80c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.221692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a091fe-45e0-4fb5-bb10-8d173de0a80c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.230450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ebe6496e-a288-430c-aef2-a7ee97443ddd None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:32.232020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ebe6496e-a288-430c-aef2-a7ee97443ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:32.240505 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 619/1240] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:32.241287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6496e-a288-430c-aef2-a7ee97443ddd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.249892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 622/1241] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/users => generated 296 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.249892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a091fe-45e0-4fb5-bb10-8d173de0a80c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.254100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6496e-a288-430c-aef2-a7ee97443ddd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.257867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a091fe-45e0-4fb5-bb10-8d173de0a80c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.259394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8cbd-688d-4fdd-b28a-f676e0cc3471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.259981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8cbd-688d-4fdd-b28a-f676e0cc3471 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.260263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8cbd-688d-4fdd-b28a-f676e0cc3471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.260528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8cbd-688d-4fdd-b28a-f676e0cc3471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.261161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b8cbd-688d-4fdd-b28a-f676e0cc3471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.262670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.262799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b8cbd-688d-4fdd-b28a-f676e0cc3471 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.263263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.263536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.263804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.264442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:32.264739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.266316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.268466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:32.273461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 620/1242] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:32.277824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8cbd-688d-4fdd-b28a-f676e0cc3471 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.279708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 623/1243] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:32.280532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.287469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8cbd-688d-4fdd-b28a-f676e0cc3471 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.291622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e06e-d3f3-4300-8d3c-8a3766095a01 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.292141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.292846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.293122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.293421 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.294099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:32.294426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.296003 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.296787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.297442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.297718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.297982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.298659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:32.299007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.300666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.309504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 621/1244] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:32.310304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.313870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 624/1245] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:32.315670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.321737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105c88d-3be3-485a-813f-8223e38070e1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.326582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1898d63-11c1-4848-9f1c-410d4acb14a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.327176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1898d63-11c1-4848-9f1c-410d4acb14a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.327282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef6968-10bb-46a0-8490-224ccf520750 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.327450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1898d63-11c1-4848-9f1c-410d4acb14a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.327716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1898d63-11c1-4848-9f1c-410d4acb14a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.328337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1898d63-11c1-4848-9f1c-410d4acb14a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.330231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1898d63-11c1-4848-9f1c-410d4acb14a1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.331850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6841ddb-08b8-4d6a-8388-2a44f4094674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.332433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6841ddb-08b8-4d6a-8388-2a44f4094674 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.332709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6841ddb-08b8-4d6a-8388-2a44f4094674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.332973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6841ddb-08b8-4d6a-8388-2a44f4094674 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.333586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6841ddb-08b8-4d6a-8388-2a44f4094674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.335487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6841ddb-08b8-4d6a-8388-2a44f4094674 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.358506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 622/1246] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.359463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1898d63-11c1-4848-9f1c-410d4acb14a1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.364270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 625/1247] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.365179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6841ddb-08b8-4d6a-8388-2a44f4094674 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.424440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1898d63-11c1-4848-9f1c-410d4acb14a1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.426578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6841ddb-08b8-4d6a-8388-2a44f4094674 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.429171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.429772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.430188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.431175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.431648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b944d-6064-4cb5-9f55-860f2462058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.431835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:32.432133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.432260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b944d-6064-4cb5-9f55-860f2462058b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.432493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b944d-6064-4cb5-9f55-860f2462058b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.432757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b944d-6064-4cb5-9f55-860f2462058b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.433417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b944d-6064-4cb5-9f55-860f2462058b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.433720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.435039 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b944d-6064-4cb5-9f55-860f2462058b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.448846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 623/1249] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:32.449027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 626/1249] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:32.449496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.449583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b944d-6064-4cb5-9f55-860f2462058b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.457362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8812d-0381-4e83-a7ed-37b592d1f7d8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.459641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b944d-6064-4cb5-9f55-860f2462058b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.462544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba83b69-f85f-4b6c-99cc-37893f5bb3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.463194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba83b69-f85f-4b6c-99cc-37893f5bb3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.463465 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba83b69-f85f-4b6c-99cc-37893f5bb3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.463749 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba83b69-f85f-4b6c-99cc-37893f5bb3f7 admin admin] PATH_INFO: `/v3/projects/6e2cba05193f429fb9ffef6f66205945/users/ca96437c5fdc4c1ba41ca7960813dfae/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.464873 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae589-250f-47f1-a071-4ddafe79e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.465466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae589-250f-47f1-a071-4ddafe79e9cc admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.465832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae589-250f-47f1-a071-4ddafe79e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.466017 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae589-250f-47f1-a071-4ddafe79e9cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.466688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecae589-250f-47f1-a071-4ddafe79e9cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.468611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecae589-250f-47f1-a071-4ddafe79e9cc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.478599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba83b69-f85f-4b6c-99cc-37893f5bb3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2cba05193f429fb9ffef6f66205945, user_id=ca96437c5fdc4c1ba41ca7960813dfae, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.481260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba83b69-f85f-4b6c-99cc-37893f5bb3f7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.495058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 624/1250] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:32 2022] PUT /identity/v3/projects/6e2cba05193f429fb9ffef6f66205945/users/ca96437c5fdc4c1ba41ca7960813dfae/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:32.495844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba83b69-f85f-4b6c-99cc-37893f5bb3f7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.498317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 627/1251] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.499179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae589-250f-47f1-a071-4ddafe79e9cc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.547908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:32.558825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba83b69-f85f-4b6c-99cc-37893f5bb3f7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.562414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae589-250f-47f1-a071-4ddafe79e9cc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.563436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb102a5-6518-4fac-8990-e8388ae9cf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.564064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb102a5-6518-4fac-8990-e8388ae9cf4d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.564339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb102a5-6518-4fac-8990-e8388ae9cf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.564608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb102a5-6518-4fac-8990-e8388ae9cf4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.565384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb102a5-6518-4fac-8990-e8388ae9cf4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.567158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69435fbb-27d4-4f69-9355-0d3451fb8d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.567345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb102a5-6518-4fac-8990-e8388ae9cf4d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.567753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69435fbb-27d4-4f69-9355-0d3451fb8d87 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.568030 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69435fbb-27d4-4f69-9355-0d3451fb8d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.568297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69435fbb-27d4-4f69-9355-0d3451fb8d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.568912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69435fbb-27d4-4f69-9355-0d3451fb8d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.571385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69435fbb-27d4-4f69-9355-0d3451fb8d87 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.595557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 625/1252] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.596443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb102a5-6518-4fac-8990-e8388ae9cf4d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.598303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 628/1253] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.599332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69435fbb-27d4-4f69-9355-0d3451fb8d87 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.623507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:32.661477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69435fbb-27d4-4f69-9355-0d3451fb8d87 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.666806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6a18a-fc03-4b2a-89af-9f76a61b5151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.667013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb102a5-6518-4fac-8990-e8388ae9cf4d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.667405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6a18a-fc03-4b2a-89af-9f76a61b5151 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.667680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6a18a-fc03-4b2a-89af-9f76a61b5151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.667941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6a18a-fc03-4b2a-89af-9f76a61b5151 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.668559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6a18a-fc03-4b2a-89af-9f76a61b5151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.670490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6a18a-fc03-4b2a-89af-9f76a61b5151 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.671681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a1fba-111d-43f1-a8ae-616ce0708f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.672268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a1fba-111d-43f1-a8ae-616ce0708f49 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.672546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a1fba-111d-43f1-a8ae-616ce0708f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.672815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a1fba-111d-43f1-a8ae-616ce0708f49 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.673463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a1fba-111d-43f1-a8ae-616ce0708f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.675399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a1fba-111d-43f1-a8ae-616ce0708f49 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.695100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 629/1254] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.695100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6a18a-fc03-4b2a-89af-9f76a61b5151 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.700475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 626/1255] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.700475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a1fba-111d-43f1-a8ae-616ce0708f49 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.703523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6a18a-fc03-4b2a-89af-9f76a61b5151 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.707958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a1fba-111d-43f1-a8ae-616ce0708f49 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.708149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dc430-f15b-451d-9c6b-9d186d140f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.708735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc430-f15b-451d-9c6b-9d186d140f3f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.709010 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc430-f15b-451d-9c6b-9d186d140f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.709280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc430-f15b-451d-9c6b-9d186d140f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.709904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dc430-f15b-451d-9c6b-9d186d140f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.711608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dc430-f15b-451d-9c6b-9d186d140f3f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.712630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01737581-ad91-4c6e-a56e-47f64232f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.713378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737581-ad91-4c6e-a56e-47f64232f0db admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.714015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737581-ad91-4c6e-a56e-47f64232f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.714292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737581-ad91-4c6e-a56e-47f64232f0db admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.714946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01737581-ad91-4c6e-a56e-47f64232f0db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.717056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01737581-ad91-4c6e-a56e-47f64232f0db admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.721466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 630/1256] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:32.723668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b72e-3ce6-4001-a126-45aa61e8328c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.724162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b72e-3ce6-4001-a126-45aa61e8328c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.724622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b72e-3ce6-4001-a126-45aa61e8328c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.744746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-40c7b72e-3ce6-4001-a126-45aa61e8328c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:32.745944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-40c7b72e-3ce6-4001-a126-45aa61e8328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:32.749623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 627/1257] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.749623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01737581-ad91-4c6e-a56e-47f64232f0db admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.806852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 631/1258] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:32.807761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7b72e-3ce6-4001-a126-45aa61e8328c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.812347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01737581-ad91-4c6e-a56e-47f64232f0db admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.816932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7b72e-3ce6-4001-a126-45aa61e8328c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.817095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb766-b3be-4c4e-9ef3-b28918ef7f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.817590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb766-b3be-4c4e-9ef3-b28918ef7f34 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.818078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb766-b3be-4c4e-9ef3-b28918ef7f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.818381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb766-b3be-4c4e-9ef3-b28918ef7f34 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.819083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bfb766-b3be-4c4e-9ef3-b28918ef7f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.821542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bfb766-b3be-4c4e-9ef3-b28918ef7f34 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.822872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9c277-62ca-4a9c-acc8-086af94183e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.823462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9c277-62ca-4a9c-acc8-086af94183e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.823735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9c277-62ca-4a9c-acc8-086af94183e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.824006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9c277-62ca-4a9c-acc8-086af94183e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.824629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9c277-62ca-4a9c-acc8-086af94183e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.826556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9c277-62ca-4a9c-acc8-086af94183e9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.844461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 628/1259] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.845374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb766-b3be-4c4e-9ef3-b28918ef7f34 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.850100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 632/1260] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.851028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9c277-62ca-4a9c-acc8-086af94183e9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.858770 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb766-b3be-4c4e-9ef3-b28918ef7f34 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.861163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9c277-62ca-4a9c-acc8-086af94183e9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.863325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dddd6-6d07-4f1e-b745-9abc48e070c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.863954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dddd6-6d07-4f1e-b745-9abc48e070c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.864232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dddd6-6d07-4f1e-b745-9abc48e070c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.864497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dddd6-6d07-4f1e-b745-9abc48e070c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.865115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dddd6-6d07-4f1e-b745-9abc48e070c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.865822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdb889-ec6b-452c-a7f9-ef9faba5ae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.866459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdb889-ec6b-452c-a7f9-ef9faba5ae70 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.866811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdb889-ec6b-452c-a7f9-ef9faba5ae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.867082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dddd6-6d07-4f1e-b745-9abc48e070c1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.867191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdb889-ec6b-452c-a7f9-ef9faba5ae70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.867719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdb889-ec6b-452c-a7f9-ef9faba5ae70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.869527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdb889-ec6b-452c-a7f9-ef9faba5ae70 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.880035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 633/1261] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:32.880881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdb889-ec6b-452c-a7f9-ef9faba5ae70 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.889612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 629/1262] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/users => generated 283 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.890423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dddd6-6d07-4f1e-b745-9abc48e070c1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.891089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdb889-ec6b-452c-a7f9-ef9faba5ae70 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.895847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda7a3e-2128-46ed-96b9-1ccf4ffa3be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.896441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda7a3e-2128-46ed-96b9-1ccf4ffa3be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.896719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda7a3e-2128-46ed-96b9-1ccf4ffa3be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.896990 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda7a3e-2128-46ed-96b9-1ccf4ffa3be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.897623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda7a3e-2128-46ed-96b9-1ccf4ffa3be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.899887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda7a3e-2128-46ed-96b9-1ccf4ffa3be3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.902120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dddd6-6d07-4f1e-b745-9abc48e070c1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.906645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a63371-e51b-4c3e-a8c4-4223d1d8e95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.907269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a63371-e51b-4c3e-a8c4-4223d1d8e95a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.907541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a63371-e51b-4c3e-a8c4-4223d1d8e95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.907807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a63371-e51b-4c3e-a8c4-4223d1d8e95a admin admin] PATH_INFO: `/v3/projects/895b2d6d988f4ab7a0d9dcf9e130af42/users/aaedb0e1920a42eb97e8dc2f3ec76905/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.912746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 634/1263] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:32.912746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda7a3e-2128-46ed-96b9-1ccf4ffa3be3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.922113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda7a3e-2128-46ed-96b9-1ccf4ffa3be3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.923648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a63371-e51b-4c3e-a8c4-4223d1d8e95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=895b2d6d988f4ab7a0d9dcf9e130af42, user_id=aaedb0e1920a42eb97e8dc2f3ec76905, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.926299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a63371-e51b-4c3e-a8c4-4223d1d8e95a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.926884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8e1e-26e4-4d35-9156-8c9968634597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:32.927467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8e1e-26e4-4d35-9156-8c9968634597 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:32.927746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8e1e-26e4-4d35-9156-8c9968634597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:32.928017 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8e1e-26e4-4d35-9156-8c9968634597 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:32.928667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c8e1e-26e4-4d35-9156-8c9968634597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:32.931177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c8e1e-26e4-4d35-9156-8c9968634597 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:32.940010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 630/1264] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:32 2022] PUT /identity/v3/projects/895b2d6d988f4ab7a0d9dcf9e130af42/users/aaedb0e1920a42eb97e8dc2f3ec76905/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:32.942484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a63371-e51b-4c3e-a8c4-4223d1d8e95a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.944524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:32.956658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 635/1265] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:32 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:32.960364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8e1e-26e4-4d35-9156-8c9968634597 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:32.970396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8e1e-26e4-4d35-9156-8c9968634597 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:32.983652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:33.001439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a63371-e51b-4c3e-a8c4-4223d1d8e95a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.006243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbd8fa-e01f-4ad6-91b7-79bec2a0b3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.006973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbd8fa-e01f-4ad6-91b7-79bec2a0b3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.007250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbd8fa-e01f-4ad6-91b7-79bec2a0b3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.007521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbd8fa-e01f-4ad6-91b7-79bec2a0b3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.028836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05c466-b04c-4bd6-9401-b9e92360d701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.029456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05c466-b04c-4bd6-9401-b9e92360d701 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.029738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05c466-b04c-4bd6-9401-b9e92360d701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.030016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05c466-b04c-4bd6-9401-b9e92360d701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.031072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05c466-b04c-4bd6-9401-b9e92360d701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.032798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05c466-b04c-4bd6-9401-b9e92360d701 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.044728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 636/1266] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.045636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05c466-b04c-4bd6-9401-b9e92360d701 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.055229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05c466-b04c-4bd6-9401-b9e92360d701 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.060507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78d82d-3db4-4ec1-9844-e75d7edb420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.061090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78d82d-3db4-4ec1-9844-e75d7edb420e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.061369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78d82d-3db4-4ec1-9844-e75d7edb420e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.061647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78d82d-3db4-4ec1-9844-e75d7edb420e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.062278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78d82d-3db4-4ec1-9844-e75d7edb420e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.063877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78d82d-3db4-4ec1-9844-e75d7edb420e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.071833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbd8fa-e01f-4ad6-91b7-79bec2a0b3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.073210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbd8fa-e01f-4ad6-91b7-79bec2a0b3f2 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.074916 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 637/1267] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.075883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78d82d-3db4-4ec1-9844-e75d7edb420e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.083768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 631/1268] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:33.084736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbd8fa-e01f-4ad6-91b7-79bec2a0b3f2 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.091712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78d82d-3db4-4ec1-9844-e75d7edb420e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.094495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbd8fa-e01f-4ad6-91b7-79bec2a0b3f2 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.096596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589dba1-37cd-415f-ac11-39fb2f5935a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.097209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589dba1-37cd-415f-ac11-39fb2f5935a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.097470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589dba1-37cd-415f-ac11-39fb2f5935a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.097740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589dba1-37cd-415f-ac11-39fb2f5935a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.098400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589dba1-37cd-415f-ac11-39fb2f5935a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.099972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589dba1-37cd-415f-ac11-39fb2f5935a6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.100116 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d3d7-348e-4727-bca0-619431d3ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.100557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d3d7-348e-4727-bca0-619431d3ab0c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.100828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d3d7-348e-4727-bca0-619431d3ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.101538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d3d7-348e-4727-bca0-619431d3ab0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.101746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66d3d7-348e-4727-bca0-619431d3ab0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.104305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66d3d7-348e-4727-bca0-619431d3ab0c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.113544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 638/1269] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.114431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589dba1-37cd-415f-ac11-39fb2f5935a6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.117451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 632/1270] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.118852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca84244-b817-4cf6-8b60-a19c3c18f0a9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.119408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca84244-b817-4cf6-8b60-a19c3c18f0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.119547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca84244-b817-4cf6-8b60-a19c3c18f0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.125231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589dba1-37cd-415f-ac11-39fb2f5935a6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.130706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c31f6-c9a6-48f2-948e-14a3fb5a4b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.131306 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c31f6-c9a6-48f2-948e-14a3fb5a4b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.131580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c31f6-c9a6-48f2-948e-14a3fb5a4b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.131851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c31f6-c9a6-48f2-948e-14a3fb5a4b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.132486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c31f6-c9a6-48f2-948e-14a3fb5a4b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.135225 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c31f6-c9a6-48f2-948e-14a3fb5a4b4e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.141249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4ca84244-b817-4cf6-8b60-a19c3c18f0a9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:33.142906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ca84244-b817-4cf6-8b60-a19c3c18f0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:33.146008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 639/1271] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.146839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c31f6-c9a6-48f2-948e-14a3fb5a4b4e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.159068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 633/1272] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:33.160336 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca84244-b817-4cf6-8b60-a19c3c18f0a9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.160650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c31f6-c9a6-48f2-948e-14a3fb5a4b4e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.166663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd363-5340-4193-88b2-725a136bcec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.167651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd363-5340-4193-88b2-725a136bcec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.168039 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd363-5340-4193-88b2-725a136bcec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.168350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd363-5340-4193-88b2-725a136bcec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.169031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd363-5340-4193-88b2-725a136bcec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.170930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd363-5340-4193-88b2-725a136bcec9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.172998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca84244-b817-4cf6-8b60-a19c3c18f0a9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.179966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59311b-aeb7-4ecd-bb81-959d79c50d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.181071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59311b-aeb7-4ecd-bb81-959d79c50d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.181322 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59311b-aeb7-4ecd-bb81-959d79c50d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.181611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59311b-aeb7-4ecd-bb81-959d79c50d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.182146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 640/1273] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.182849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd363-5340-4193-88b2-725a136bcec9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.185673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59311b-aeb7-4ecd-bb81-959d79c50d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.187881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59311b-aeb7-4ecd-bb81-959d79c50d94 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.193424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd363-5340-4193-88b2-725a136bcec9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.199037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eaf0b-c5fe-46ad-9908-73c5b19d2286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.199713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eaf0b-c5fe-46ad-9908-73c5b19d2286 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.199913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eaf0b-c5fe-46ad-9908-73c5b19d2286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.200183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eaf0b-c5fe-46ad-9908-73c5b19d2286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.200277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 634/1274] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.200823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eaf0b-c5fe-46ad-9908-73c5b19d2286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.201324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59311b-aeb7-4ecd-bb81-959d79c50d94 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.202436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eaf0b-c5fe-46ad-9908-73c5b19d2286 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.212890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59311b-aeb7-4ecd-bb81-959d79c50d94 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.217392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 641/1275] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.217392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eaf0b-c5fe-46ad-9908-73c5b19d2286 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.220160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.220923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.221277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.221678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] PATH_INFO: `/v3/projects/fb2e0425c492454cb57df643f58fb959/users/5bddc8c303bd4c67bffbbeb68ead5f51/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.230265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eaf0b-c5fe-46ad-9908-73c5b19d2286 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.235321 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e954a3b-a759-4f3f-89b0-0af37ac8e58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.236095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e954a3b-a759-4f3f-89b0-0af37ac8e58a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.236473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e954a3b-a759-4f3f-89b0-0af37ac8e58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.236841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e954a3b-a759-4f3f-89b0-0af37ac8e58a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.237636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e954a3b-a759-4f3f-89b0-0af37ac8e58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.239405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e954a3b-a759-4f3f-89b0-0af37ac8e58a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.244691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2e0425c492454cb57df643f58fb959, user_id=5bddc8c303bd4c67bffbbeb68ead5f51, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.248417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.252382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 642/1276] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.253316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e954a3b-a759-4f3f-89b0-0af37ac8e58a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.265493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e954a3b-a759-4f3f-89b0-0af37ac8e58a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.269785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 635/1277] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:33 2022] PUT /identity/v3/projects/fb2e0425c492454cb57df643f58fb959/users/5bddc8c303bd4c67bffbbeb68ead5f51/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:33.270655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.272230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:33.340660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382538-e2d1-45e8-9a47-12b7d539ebe5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.345706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07788c-423f-4c5f-9a5d-8946dfeb95c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.346528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07788c-423f-4c5f-9a5d-8946dfeb95c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.346809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07788c-423f-4c5f-9a5d-8946dfeb95c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.347081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07788c-423f-4c5f-9a5d-8946dfeb95c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.353091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43255-3600-4d9f-82be-2376576d17f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.354289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43255-3600-4d9f-82be-2376576d17f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.354823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43255-3600-4d9f-82be-2376576d17f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.355325 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43255-3600-4d9f-82be-2376576d17f9 admin admin] PATH_INFO: `/v3/projects/9ad3932c3fcc4d42b77b26f889ac694f/users/eb7c6da9fd3c4b72a687daa923d07918/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.373433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b43255-3600-4d9f-82be-2376576d17f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad3932c3fcc4d42b77b26f889ac694f, user_id=eb7c6da9fd3c4b72a687daa923d07918, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.376662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b43255-3600-4d9f-82be-2376576d17f9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.395092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 643/1278] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:33 2022] PUT /identity/v3/projects/9ad3932c3fcc4d42b77b26f889ac694f/users/eb7c6da9fd3c4b72a687daa923d07918/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:33.395996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43255-3600-4d9f-82be-2376576d17f9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.405128 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:33.408979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:33.428221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07788c-423f-4c5f-9a5d-8946dfeb95c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.431869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07788c-423f-4c5f-9a5d-8946dfeb95c7 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.443607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 636/1279] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:33.444433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07788c-423f-4c5f-9a5d-8946dfeb95c7 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.454685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07788c-423f-4c5f-9a5d-8946dfeb95c7 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.486112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43255-3600-4d9f-82be-2376576d17f9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.491712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2324c9-cd58-4ec3-b28a-63d5a5928131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.492573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2324c9-cd58-4ec3-b28a-63d5a5928131 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.492912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2324c9-cd58-4ec3-b28a-63d5a5928131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.493239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2324c9-cd58-4ec3-b28a-63d5a5928131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.493925 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2324c9-cd58-4ec3-b28a-63d5a5928131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.495876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2324c9-cd58-4ec3-b28a-63d5a5928131 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.513881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 644/1280] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:33.514800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2324c9-cd58-4ec3-b28a-63d5a5928131 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.525686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2324c9-cd58-4ec3-b28a-63d5a5928131 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.531113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd396e-7792-44da-a31d-393e1fcbff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.532052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd396e-7792-44da-a31d-393e1fcbff8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.532344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd396e-7792-44da-a31d-393e1fcbff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.532626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd396e-7792-44da-a31d-393e1fcbff8f admin admin] PATH_INFO: `/v3/projects/9fc4d931061d4aaba3dc577413f06866/users/57f7edfdf0324725aefe7bcc3c10405a/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.551964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd396e-7792-44da-a31d-393e1fcbff8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc4d931061d4aaba3dc577413f06866, user_id=57f7edfdf0324725aefe7bcc3c10405a, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.555104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd396e-7792-44da-a31d-393e1fcbff8f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.582784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c0efe-20d8-4acb-a57a-58fd7dfd470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.583657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c0efe-20d8-4acb-a57a-58fd7dfd470f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.583922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 645/1281] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:33 2022] PUT /identity/v3/projects/9fc4d931061d4aaba3dc577413f06866/users/57f7edfdf0324725aefe7bcc3c10405a/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:33.584038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c0efe-20d8-4acb-a57a-58fd7dfd470f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.584354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c0efe-20d8-4acb-a57a-58fd7dfd470f admin admin] PATH_INFO: `/v3/projects/41a236cfbaf14211a6a3674f3c2f3a67/users/6b4b2700c20946ea93a1d20d7dea929f/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.584858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd396e-7792-44da-a31d-393e1fcbff8f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.608951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c0efe-20d8-4acb-a57a-58fd7dfd470f admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a236cfbaf14211a6a3674f3c2f3a67, user_id=6b4b2700c20946ea93a1d20d7dea929f, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.612177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c0efe-20d8-4acb-a57a-58fd7dfd470f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.635138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 637/1282] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:33 2022] PUT /identity/v3/projects/41a236cfbaf14211a6a3674f3c2f3a67/users/6b4b2700c20946ea93a1d20d7dea929f/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:33.635138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd8daf-5a1c-4855-9e7e-03a5cd0be1c6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.635138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd8daf-5a1c-4855-9e7e-03a5cd0be1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.635138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd8daf-5a1c-4855-9e7e-03a5cd0be1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.667948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ecdd8daf-5a1c-4855-9e7e-03a5cd0be1c6 None None] MFA Rules not processed for user `5bddc8c303bd4c67bffbbeb68ead5f51`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:33.669356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ecdd8daf-5a1c-4855-9e7e-03a5cd0be1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:33.678078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd396e-7792-44da-a31d-393e1fcbff8f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.754495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 638/1283] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:33.755961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ef90d-a51e-4bbd-804c-ee365919bcc5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.756258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ef90d-a51e-4bbd-804c-ee365919bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.756512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ef90d-a51e-4bbd-804c-ee365919bcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.759570 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c17e68-837c-41e4-ae6d-bc317974ffc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.760125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c17e68-837c-41e4-ae6d-bc317974ffc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.760425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c17e68-837c-41e4-ae6d-bc317974ffc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.760773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c17e68-837c-41e4-ae6d-bc317974ffc0 admin admin] PATH_INFO: `/v3/projects/0841a46c829c45a29392017401b16ece/users/7bc995383f9e47f5903ddf0c260c01fd/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.780791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ee2ef90d-a51e-4bbd-804c-ee365919bcc5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:33.782237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee2ef90d-a51e-4bbd-804c-ee365919bcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:33.787354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c17e68-837c-41e4-ae6d-bc317974ffc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0841a46c829c45a29392017401b16ece, user_id=7bc995383f9e47f5903ddf0c260c01fd, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.797068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 639/1284] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:33.797068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ef90d-a51e-4bbd-804c-ee365919bcc5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:33.797359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c17e68-837c-41e4-ae6d-bc317974ffc0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.809456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ef90d-a51e-4bbd-804c-ee365919bcc5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:33.823931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 646/1285] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:33 2022] PUT /identity/v3/projects/0841a46c829c45a29392017401b16ece/users/7bc995383f9e47f5903ddf0c260c01fd/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:33.826876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf621f7-51ea-44db-b3de-020f3f8bb796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:33.828740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf621f7-51ea-44db-b3de-020f3f8bb796 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.829293 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf621f7-51ea-44db-b3de-020f3f8bb796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.830814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf621f7-51ea-44db-b3de-020f3f8bb796 admin admin] PATH_INFO: `/v3/projects/b0620f77ac274727b9995b261106e917/users/e0a4f6af883b4af9874fc1cebe904086/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.834659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6444823-d2bd-4e9f-ac2b-41c89d50990c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.834955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6444823-d2bd-4e9f-ac2b-41c89d50990c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.835203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6444823-d2bd-4e9f-ac2b-41c89d50990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.860252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:33.861686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf621f7-51ea-44db-b3de-020f3f8bb796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0620f77ac274727b9995b261106e917, user_id=e0a4f6af883b4af9874fc1cebe904086, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:33.864780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf621f7-51ea-44db-b3de-020f3f8bb796 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:33.874254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b6444823-d2bd-4e9f-ac2b-41c89d50990c None None] MFA Rules not processed for user `57f7edfdf0324725aefe7bcc3c10405a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:33.876157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b6444823-d2bd-4e9f-ac2b-41c89d50990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:33.883200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 640/1286] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:33 2022] PUT /identity/v3/projects/b0620f77ac274727b9995b261106e917/users/e0a4f6af883b4af9874fc1cebe904086/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:33.884781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b82b-9c92-41a7-b193-d5aaa9f00591 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:33.885070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b82b-9c92-41a7-b193-d5aaa9f00591 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:33.885342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b82b-9c92-41a7-b193-d5aaa9f00591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:33.909522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-02f5b82b-9c92-41a7-b193-d5aaa9f00591 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:33.911219 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-02f5b82b-9c92-41a7-b193-d5aaa9f00591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:33.941410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:33.997235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 647/1287] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:33.998129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6444823-d2bd-4e9f-ac2b-41c89d50990c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.029192 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 641/1288] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:34.030029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5b82b-9c92-41a7-b193-d5aaa9f00591 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.145080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6444823-d2bd-4e9f-ac2b-41c89d50990c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:34.153218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4745f-4d5c-4643-82c3-dcf5b5fd6931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:34.154001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4745f-4d5c-4643-82c3-dcf5b5fd6931 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.155169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4745f-4d5c-4643-82c3-dcf5b5fd6931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.155169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4745f-4d5c-4643-82c3-dcf5b5fd6931 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.157308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4745f-4d5c-4643-82c3-dcf5b5fd6931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:34.160100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4745f-4d5c-4643-82c3-dcf5b5fd6931 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:34.177311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5b82b-9c92-41a7-b193-d5aaa9f00591 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:34.192754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3c8ad-3c5b-497f-9985-4982f472d7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:34.193380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3c8ad-3c5b-497f-9985-4982f472d7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.193661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3c8ad-3c5b-497f-9985-4982f472d7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.193933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3c8ad-3c5b-497f-9985-4982f472d7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.203026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 648/1289] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:33 2022] POST /identity/v3/projects => generated 363 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:34.203026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49292f3-f100-413b-9e15-5dc75f11cfdd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.203026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49292f3-f100-413b-9e15-5dc75f11cfdd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.203381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49292f3-f100-413b-9e15-5dc75f11cfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.213661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3c8ad-3c5b-497f-9985-4982f472d7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:34.215621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3c8ad-3c5b-497f-9985-4982f472d7d6 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:34.236842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 642/1290] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:34.238238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bc91-b5ee-4fec-8530-537190288653 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.238557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bc91-b5ee-4fec-8530-537190288653 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.238831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bc91-b5ee-4fec-8530-537190288653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.271151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c49292f3-f100-413b-9e15-5dc75f11cfdd None None] MFA Rules not processed for user `7bc995383f9e47f5903ddf0c260c01fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:34.273204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c49292f3-f100-413b-9e15-5dc75f11cfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:34.281549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7924bc91-b5ee-4fec-8530-537190288653 None None] MFA Rules not processed for user `e0a4f6af883b4af9874fc1cebe904086`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:34.282925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7924bc91-b5ee-4fec-8530-537190288653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:34.371278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 649/1291] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:34.371278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02415f5c-77e6-45cb-b4b9-712d06be7bef None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.371278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02415f5c-77e6-45cb-b4b9-712d06be7bef None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.371278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02415f5c-77e6-45cb-b4b9-712d06be7bef None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.381396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 650/1292] 10.222.0.16 () {54 vars in 961 bytes} [Mon Oct 17 10:24:34 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:24:34.382284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02415f5c-77e6-45cb-b4b9-712d06be7bef None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.394617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 643/1293] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:34.395914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924bc91-b5ee-4fec-8530-537190288653 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.399731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:34.471648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02415f5c-77e6-45cb-b4b9-712d06be7bef None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:34.483061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77932fd-81a9-4c45-98fb-365617f514aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:34.483992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77932fd-81a9-4c45-98fb-365617f514aa service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.484335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77932fd-81a9-4c45-98fb-365617f514aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.484603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77932fd-81a9-4c45-98fb-365617f514aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.499300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:34.506270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924bc91-b5ee-4fec-8530-537190288653 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:34.512612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368a904-f2e7-4aba-97b8-f59bce82e2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:34.514180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a904-f2e7-4aba-97b8-f59bce82e2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.514648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a904-f2e7-4aba-97b8-f59bce82e2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.516273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a904-f2e7-4aba-97b8-f59bce82e2a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.517198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368a904-f2e7-4aba-97b8-f59bce82e2a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:34.519842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368a904-f2e7-4aba-97b8-f59bce82e2a5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:34.562241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 644/1294] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:34 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:34.563363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368a904-f2e7-4aba-97b8-f59bce82e2a5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.582196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77932fd-81a9-4c45-98fb-365617f514aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:34.584728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77932fd-81a9-4c45-98fb-365617f514aa service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:34.596928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 651/1295] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:34.598626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a91ed-8252-498c-b61d-fc96c4489b3e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.599060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a91ed-8252-498c-b61d-fc96c4489b3e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.599829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a91ed-8252-498c-b61d-fc96c4489b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.647271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f77a91ed-8252-498c-b61d-fc96c4489b3e None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:34.649694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f77a91ed-8252-498c-b61d-fc96c4489b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:34.697914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368a904-f2e7-4aba-97b8-f59bce82e2a5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:34.708474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eecbfa-acef-4715-b1e6-408d521b9b12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:34.709138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eecbfa-acef-4715-b1e6-408d521b9b12 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.709412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eecbfa-acef-4715-b1e6-408d521b9b12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.709679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eecbfa-acef-4715-b1e6-408d521b9b12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.728672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eecbfa-acef-4715-b1e6-408d521b9b12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:34.730123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eecbfa-acef-4715-b1e6-408d521b9b12 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:34.760038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 645/1296] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:34.763627 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eecbfa-acef-4715-b1e6-408d521b9b12 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.788539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eecbfa-acef-4715-b1e6-408d521b9b12 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:34.788893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 652/1297] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Oct 17 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:34.790457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a91ed-8252-498c-b61d-fc96c4489b3e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.803224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cae3f-a396-48ff-b609-a2a53c4f495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:34.804116 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cae3f-a396-48ff-b609-a2a53c4f495a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.804569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cae3f-a396-48ff-b609-a2a53c4f495a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.806073 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cae3f-a396-48ff-b609-a2a53c4f495a admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.806948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cae3f-a396-48ff-b609-a2a53c4f495a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:34.810851 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cae3f-a396-48ff-b609-a2a53c4f495a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:34.817875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a91ed-8252-498c-b61d-fc96c4489b3e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:34.828861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2b949-5915-41ab-a0ec-6489240e4e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:34.829489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b949-5915-41ab-a0ec-6489240e4e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:34.829763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b949-5915-41ab-a0ec-6489240e4e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:34.830316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b949-5915-41ab-a0ec-6489240e4e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:34.831294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2b949-5915-41ab-a0ec-6489240e4e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:34.832929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2b949-5915-41ab-a0ec-6489240e4e37 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:34.860591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 653/1298] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:34.861419 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2b949-5915-41ab-a0ec-6489240e4e37 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.885499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 646/1299] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:34 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:34.886797 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cae3f-a396-48ff-b609-a2a53c4f495a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:34.893631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2b949-5915-41ab-a0ec-6489240e4e37 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:34.957995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:35.033075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cae3f-a396-48ff-b609-a2a53c4f495a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:35.036283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87da365-94bb-44aa-b5b1-f74a743c9805 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:35.037014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da365-94bb-44aa-b5b1-f74a743c9805 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:35.037492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da365-94bb-44aa-b5b1-f74a743c9805 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:35.037646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da365-94bb-44aa-b5b1-f74a743c9805 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:35.046805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65da1b5-876c-47e4-a73e-a1d7e3b4d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:35.047498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65da1b5-876c-47e4-a73e-a1d7e3b4d6af admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:35.049168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65da1b5-876c-47e4-a73e-a1d7e3b4d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:35.049451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65da1b5-876c-47e4-a73e-a1d7e3b4d6af admin admin] PATH_INFO: `/v3/projects/d921d8edfabf4a39934705b40d20c0c8/users/35acaf1f6c4548a4a95ff42882ab4bc7/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:35.094462 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65da1b5-876c-47e4-a73e-a1d7e3b4d6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=d921d8edfabf4a39934705b40d20c0c8, user_id=35acaf1f6c4548a4a95ff42882ab4bc7, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:35.098990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65da1b5-876c-47e4-a73e-a1d7e3b4d6af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:35.128844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 647/1300] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:34 2022] PUT /identity/v3/projects/d921d8edfabf4a39934705b40d20c0c8/users/35acaf1f6c4548a4a95ff42882ab4bc7/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:35.133069 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65da1b5-876c-47e4-a73e-a1d7e3b4d6af admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:35.171599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87da365-94bb-44aa-b5b1-f74a743c9805 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:35.173520 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87da365-94bb-44aa-b5b1-f74a743c9805 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:35.194198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 654/1301] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:35.195131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87da365-94bb-44aa-b5b1-f74a743c9805 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:35.220301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87da365-94bb-44aa-b5b1-f74a743c9805 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:35.266985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:35.293020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65da1b5-876c-47e4-a73e-a1d7e3b4d6af admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:35.307290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c2a7e-69d7-4b66-8ebe-e58f2a379368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:35.309957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c2a7e-69d7-4b66-8ebe-e58f2a379368 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:35.310244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c2a7e-69d7-4b66-8ebe-e58f2a379368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:35.310543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c2a7e-69d7-4b66-8ebe-e58f2a379368 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:35.313438 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c2a7e-69d7-4b66-8ebe-e58f2a379368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:35.320566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c2a7e-69d7-4b66-8ebe-e58f2a379368 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:35.346914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad36e4-0968-4da5-abd4-4075b806ed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:35.347565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad36e4-0968-4da5-abd4-4075b806ed3e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:35.347842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad36e4-0968-4da5-abd4-4075b806ed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:35.348246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad36e4-0968-4da5-abd4-4075b806ed3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:35.348946 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad36e4-0968-4da5-abd4-4075b806ed3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:35.353280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad36e4-0968-4da5-abd4-4075b806ed3e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:35.366537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 648/1302] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:35 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:35.369963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c2a7e-69d7-4b66-8ebe-e58f2a379368 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:35.387423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 655/1303] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:35.388332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad36e4-0968-4da5-abd4-4075b806ed3e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:35.396966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c2a7e-69d7-4b66-8ebe-e58f2a379368 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:35.403798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad36e4-0968-4da5-abd4-4075b806ed3e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:35.410866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de832597-64ee-43f3-a0a7-b43531a682a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:35.411807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de832597-64ee-43f3-a0a7-b43531a682a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:35.412258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de832597-64ee-43f3-a0a7-b43531a682a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:35.412697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de832597-64ee-43f3-a0a7-b43531a682a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:35.413517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de832597-64ee-43f3-a0a7-b43531a682a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:35.420196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de832597-64ee-43f3-a0a7-b43531a682a8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:35.422311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc614348-1f5f-4223-8147-0e6149c39ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:35.423460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc614348-1f5f-4223-8147-0e6149c39ece admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:35.423996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc614348-1f5f-4223-8147-0e6149c39ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:35.424651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc614348-1f5f-4223-8147-0e6149c39ece admin admin] PATH_INFO: `/v3/projects/d921d8edfabf4a39934705b40d20c0c8/users/35acaf1f6c4548a4a95ff42882ab4bc7/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:35.433951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc614348-1f5f-4223-8147-0e6149c39ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=d921d8edfabf4a39934705b40d20c0c8, user_id=35acaf1f6c4548a4a95ff42882ab4bc7, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:35.438176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc614348-1f5f-4223-8147-0e6149c39ece admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:35.452217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 649/1304] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:35.456394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de832597-64ee-43f3-a0a7-b43531a682a8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:35.471971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 656/1305] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:35 2022] PUT /identity/v3/projects/d921d8edfabf4a39934705b40d20c0c8/users/35acaf1f6c4548a4a95ff42882ab4bc7/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:35.473466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6add96b-2dc0-4502-b073-b70993d88434 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:35.473752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6add96b-2dc0-4502-b073-b70993d88434 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:35.474025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6add96b-2dc0-4502-b073-b70993d88434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:35.482672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de832597-64ee-43f3-a0a7-b43531a682a8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:35.543877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d6add96b-2dc0-4502-b073-b70993d88434 None None] MFA Rules not processed for user `35acaf1f6c4548a4a95ff42882ab4bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:35.545026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d6add96b-2dc0-4502-b073-b70993d88434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:35.643044 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:35.669133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77085c0-dbca-4dfa-bbe4-cab5a937555d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:35.671810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77085c0-dbca-4dfa-bbe4-cab5a937555d admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:35.672124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77085c0-dbca-4dfa-bbe4-cab5a937555d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:35.672392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77085c0-dbca-4dfa-bbe4-cab5a937555d admin admin] PATH_INFO: `/v3/projects/00fff1386985436b8addad058376ff66/users/2092aa7cb2df4f9589a48675e538d3cd/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:35.729672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 657/1306] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:35.729927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77085c0-dbca-4dfa-bbe4-cab5a937555d admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fff1386985436b8addad058376ff66, user_id=2092aa7cb2df4f9589a48675e538d3cd, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:35.738260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77085c0-dbca-4dfa-bbe4-cab5a937555d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:35.740576 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6add96b-2dc0-4502-b073-b70993d88434 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:35.779516 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 650/1307] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:35 2022] PUT /identity/v3/projects/00fff1386985436b8addad058376ff66/users/2092aa7cb2df4f9589a48675e538d3cd/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:35.783412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77085c0-dbca-4dfa-bbe4-cab5a937555d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:35.896499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6add96b-2dc0-4502-b073-b70993d88434 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:36.034239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77085c0-dbca-4dfa-bbe4-cab5a937555d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:36.053715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322b8dd-a1e9-4b47-937e-30d8519ba402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:36.054427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b8dd-a1e9-4b47-937e-30d8519ba402 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:36.054754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b8dd-a1e9-4b47-937e-30d8519ba402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:36.055021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b8dd-a1e9-4b47-937e-30d8519ba402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:36.057480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322b8dd-a1e9-4b47-937e-30d8519ba402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:36.061691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322b8dd-a1e9-4b47-937e-30d8519ba402 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:36.138631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457d7bd-c4f5-47db-98ed-7c153ed62234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:36.138631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457d7bd-c4f5-47db-98ed-7c153ed62234 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:36.138631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457d7bd-c4f5-47db-98ed-7c153ed62234 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:36.138631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457d7bd-c4f5-47db-98ed-7c153ed62234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:36.166659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 651/1308] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:36.166659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb52a1-0172-40c8-8010-c10949f8cdbd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:36.166659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb52a1-0172-40c8-8010-c10949f8cdbd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:36.166659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb52a1-0172-40c8-8010-c10949f8cdbd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:36.166659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 652/1309] 10.222.0.16 () {54 vars in 912 bytes} [Mon Oct 17 10:24:36 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:24:36.169995 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb52a1-0172-40c8-8010-c10949f8cdbd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:36.200525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb52a1-0172-40c8-8010-c10949f8cdbd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:36.213503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1b234-672d-466b-a831-3fb2239cbcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:36.214464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1b234-672d-466b-a831-3fb2239cbcd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:36.214743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1b234-672d-466b-a831-3fb2239cbcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:36.215012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1b234-672d-466b-a831-3fb2239cbcd5 admin admin] PATH_INFO: `/v3/projects/00fff1386985436b8addad058376ff66/users/2092aa7cb2df4f9589a48675e538d3cd/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:36.219409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1b234-672d-466b-a831-3fb2239cbcd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fff1386985436b8addad058376ff66, user_id=2092aa7cb2df4f9589a48675e538d3cd, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:36.222082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1b234-672d-466b-a831-3fb2239cbcd5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:36.264355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 653/1310] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:36 2022] PUT /identity/v3/projects/00fff1386985436b8addad058376ff66/users/2092aa7cb2df4f9589a48675e538d3cd/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:36.264355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40127a-612c-4239-9815-afd92fac1b9a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:36.264355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40127a-612c-4239-9815-afd92fac1b9a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:36.264355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40127a-612c-4239-9815-afd92fac1b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:36.297486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457d7bd-c4f5-47db-98ed-7c153ed62234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:36.301058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457d7bd-c4f5-47db-98ed-7c153ed62234 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:36.313956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8d40127a-612c-4239-9815-afd92fac1b9a None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:36.317953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8d40127a-612c-4239-9815-afd92fac1b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:36.351288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 658/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:36.360626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b126f6-a9ef-42cf-a10f-e2ba7a1d0125 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:36.360981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b126f6-a9ef-42cf-a10f-e2ba7a1d0125 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:36.362052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b126f6-a9ef-42cf-a10f-e2ba7a1d0125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:36.427738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-56b126f6-a9ef-42cf-a10f-e2ba7a1d0125 None None] MFA Rules not processed for user `2092aa7cb2df4f9589a48675e538d3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:36.429018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-56b126f6-a9ef-42cf-a10f-e2ba7a1d0125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:36.447008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:36.529746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:36.562689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 654/1312] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:24:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:36.564821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40127a-612c-4239-9815-afd92fac1b9a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:36.624566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 659/1313] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:36.624566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b126f6-a9ef-42cf-a10f-e2ba7a1d0125 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:36.706047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40127a-612c-4239-9815-afd92fac1b9a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:36.716326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7e880-85cf-4fda-a655-0d32e5cbd878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:36.717755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7e880-85cf-4fda-a655-0d32e5cbd878 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:36.717906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7e880-85cf-4fda-a655-0d32e5cbd878 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:36.718097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7e880-85cf-4fda-a655-0d32e5cbd878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:36.808238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b126f6-a9ef-42cf-a10f-e2ba7a1d0125 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:36.812374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d1c49-1e73-41c9-b1f4-34654fe3b202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:36.813443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1c49-1e73-41c9-b1f4-34654fe3b202 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:36.813820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1c49-1e73-41c9-b1f4-34654fe3b202 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:36.814213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1c49-1e73-41c9-b1f4-34654fe3b202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:36.840462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d1c49-1e73-41c9-b1f4-34654fe3b202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:36.844546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d1c49-1e73-41c9-b1f4-34654fe3b202 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:36.887565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 660/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:36.904683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7e880-85cf-4fda-a655-0d32e5cbd878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:36.906915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7e880-85cf-4fda-a655-0d32e5cbd878 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:36.930851 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 655/1315] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:37.459206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d1c49-1e73-41c9-b1f4-34654fe3b202 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:37.481006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d1c49-1e73-41c9-b1f4-34654fe3b202 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:37.501787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe9dc6-6f7f-4408-b38c-e436fc64b9d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:37.502454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9dc6-6f7f-4408-b38c-e436fc64b9d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:37.502734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9dc6-6f7f-4408-b38c-e436fc64b9d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:37.503003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9dc6-6f7f-4408-b38c-e436fc64b9d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:37.537874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe9dc6-6f7f-4408-b38c-e436fc64b9d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:37.543572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe9dc6-6f7f-4408-b38c-e436fc64b9d0 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:37.578939 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 661/1316] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:38.417694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c99a9-cbc2-45a8-bf3e-9eb8bf89f2d8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:38.418307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c99a9-cbc2-45a8-bf3e-9eb8bf89f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:38.418307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c99a9-cbc2-45a8-bf3e-9eb8bf89f2d8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:38.419907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 656/1317] 10.222.0.16 () {54 vars in 961 bytes} [Mon Oct 17 10:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:24:38.472768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d17ad0-eeb4-48ac-a0c2-52b5160905b8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:38.473271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d17ad0-eeb4-48ac-a0c2-52b5160905b8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:38.473579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d17ad0-eeb4-48ac-a0c2-52b5160905b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:38.504663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-73d17ad0-eeb4-48ac-a0c2-52b5160905b8 None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:38.506211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-73d17ad0-eeb4-48ac-a0c2-52b5160905b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:38.518354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 662/1318] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Oct 17 10:24:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:38.525187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3e920-23c9-4d0e-9766-0eae7e48ef8c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:38.525857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3e920-23c9-4d0e-9766-0eae7e48ef8c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:38.526372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3e920-23c9-4d0e-9766-0eae7e48ef8c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:38.528313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 657/1319] 10.222.0.16 () {54 vars in 912 bytes} [Mon Oct 17 10:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:24:38.534573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2714a-222b-47bc-8009-7162265434fe None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:38.535192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2714a-222b-47bc-8009-7162265434fe None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:38.535688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2714a-222b-47bc-8009-7162265434fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:38.611288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cce2714a-222b-47bc-8009-7162265434fe None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:38.619539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cce2714a-222b-47bc-8009-7162265434fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:38.644659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 663/1320] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:24:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:38.666520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3e920-23c9-4d0e-9766-0eae7e48ef8c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:38.695684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3e920-23c9-4d0e-9766-0eae7e48ef8c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:38.705871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ecc75-2aa8-47dc-a1a3-e5d8737a7d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:38.706867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ecc75-2aa8-47dc-a1a3-e5d8737a7d18 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:38.707431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ecc75-2aa8-47dc-a1a3-e5d8737a7d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:38.707805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ecc75-2aa8-47dc-a1a3-e5d8737a7d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:38.733585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ecc75-2aa8-47dc-a1a3-e5d8737a7d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:38.735326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ecc75-2aa8-47dc-a1a3-e5d8737a7d18 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:38.754648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 658/1321] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:39.497074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2714a-222b-47bc-8009-7162265434fe None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:39.719870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2714a-222b-47bc-8009-7162265434fe None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:39.726756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:39.728999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:39.729695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:39.730148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] PATH_INFO: `/v3/users/e0a4f6af883b4af9874fc1cebe904086` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:39.755445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a4f6af883b4af9874fc1cebe904086)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:39.756891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:24:39.756891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Oct 17 10:24:39.756891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: Oct 17 10:24:39.760752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:39.948674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:39.955983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a4f6af883b4af9874fc1cebe904086'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:39.956905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a4f6af883b4af9874fc1cebe904086'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:40.005779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a4f6af883b4af9874fc1cebe904086'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:40.027895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a4f6af883b4af9874fc1cebe904086'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:40.029100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a4f6af883b4af9874fc1cebe904086'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:40.063459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397992b-8fc6-4771-aafe-e8e7b26b1ce0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:40.064342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397992b-8fc6-4771-aafe-e8e7b26b1ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:40.064888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397992b-8fc6-4771-aafe-e8e7b26b1ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:40.067774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 664/1322] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:24:39 2022] DELETE /identity/v3/users/e0a4f6af883b4af9874fc1cebe904086 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:40.091291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2397992b-8fc6-4771-aafe-e8e7b26b1ce0 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:40.092564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2397992b-8fc6-4771-aafe-e8e7b26b1ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:40.298984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 659/1323] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:40.365679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:40.368860 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:40.479483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa168b8-0abd-4ed8-84fa-88994bd88cdc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:40.493595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45acc-ab3e-4bc9-a17b-e7a65052e7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:40.494934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45acc-ab3e-4bc9-a17b-e7a65052e7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:40.495487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45acc-ab3e-4bc9-a17b-e7a65052e7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:40.495967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45acc-ab3e-4bc9-a17b-e7a65052e7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:40.523434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45acc-ab3e-4bc9-a17b-e7a65052e7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:40.527016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45acc-ab3e-4bc9-a17b-e7a65052e7d6 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:40.545047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 665/1324] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:41.356836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397992b-8fc6-4771-aafe-e8e7b26b1ce0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:41.356836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2397992b-8fc6-4771-aafe-e8e7b26b1ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:41.376363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397992b-8fc6-4771-aafe-e8e7b26b1ce0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:41.391957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:41.394622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:41.397519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:41.400858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] PATH_INFO: `/v3/users/57f7edfdf0324725aefe7bcc3c10405a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:41.404558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57f7edfdf0324725aefe7bcc3c10405a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:41.407353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:41.415021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:41.586821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57f7edfdf0324725aefe7bcc3c10405a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:41.593977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57f7edfdf0324725aefe7bcc3c10405a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:41.618609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57f7edfdf0324725aefe7bcc3c10405a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:41.662661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57f7edfdf0324725aefe7bcc3c10405a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:41.663706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57f7edfdf0324725aefe7bcc3c10405a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:41.706827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 660/1325] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:24:41 2022] DELETE /identity/v3/users/57f7edfdf0324725aefe7bcc3c10405a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:41.712059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03f8f-911d-4f28-a41b-a807603689c6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:41.712565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03f8f-911d-4f28-a41b-a807603689c6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:41.712652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03f8f-911d-4f28-a41b-a807603689c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:41.766865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c2d03f8f-911d-4f28-a41b-a807603689c6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:41.768326 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c2d03f8f-911d-4f28-a41b-a807603689c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:41.900840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 666/1326] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:42.078523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:42.081953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:42.243911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27821060-4024-4eb0-ba39-d449467a0c25 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:42.251886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537deeb-48d5-4577-9bdc-ae910c7554ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:42.252721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537deeb-48d5-4577-9bdc-ae910c7554ea service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:42.255200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537deeb-48d5-4577-9bdc-ae910c7554ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:42.255791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537deeb-48d5-4577-9bdc-ae910c7554ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:42.282090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537deeb-48d5-4577-9bdc-ae910c7554ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:42.284081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537deeb-48d5-4577-9bdc-ae910c7554ea service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:42.313241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 661/1327] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:43.949059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5bd6b-1d7f-47f7-97df-99ad280827ad None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:43.949905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5bd6b-1d7f-47f7-97df-99ad280827ad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:43.950356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5bd6b-1d7f-47f7-97df-99ad280827ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:44.024699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c2f5bd6b-1d7f-47f7-97df-99ad280827ad None None] MFA Rules not processed for user `aaedb0e1920a42eb97e8dc2f3ec76905`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:44.038096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c2f5bd6b-1d7f-47f7-97df-99ad280827ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:44.259989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:44.303856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 667/1328] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:44.315296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a539-fa4e-4761-8856-a765dff323ba None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:44.315883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a539-fa4e-4761-8856-a765dff323ba None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:44.315883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a539-fa4e-4761-8856-a765dff323ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:44.375785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3cb5a539-fa4e-4761-8856-a765dff323ba None None] MFA Rules not processed for user `aaedb0e1920a42eb97e8dc2f3ec76905`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:44.377993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3cb5a539-fa4e-4761-8856-a765dff323ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:44.401234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 662/1329] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:44.411464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5bd6b-1d7f-47f7-97df-99ad280827ad None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:44.413227 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c2f5bd6b-1d7f-47f7-97df-99ad280827ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:44.566641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5bd6b-1d7f-47f7-97df-99ad280827ad None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:44.580973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:44.581872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:44.582219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:44.582529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:44.606080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:44.607780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:44.626790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 668/1330] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:44.745375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a539-fa4e-4761-8856-a765dff323ba None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:44.746976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3cb5a539-fa4e-4761-8856-a765dff323ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:44.768196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:44.769863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:44.776493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a539-fa4e-4761-8856-a765dff323ba None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:44.799429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:44.800750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:44.801305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:44.801728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] PATH_INFO: `/v3/projects/9fc4d931061d4aaba3dc577413f06866` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:44.808461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc4d931061d4aaba3dc577413f06866)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:44.809401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92de62f-eff0-4e91-8c92-2e2f8d16a1f8 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:44.810524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 10:24:44.810524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Oct 17 10:24:44.810524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: Oct 17 10:24:44.814754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:44.826035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:44.828515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:44.828801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:44.829078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] PATH_INFO: `/v3/projects/b0620f77ac274727b9995b261106e917` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:44.836112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0620f77ac274727b9995b261106e917)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:44.845995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:45.000212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0620f77ac274727b9995b261106e917', 'request_id': 'req-72a935e3-33b2-4709-b9b8-f5eef5e6b023'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.020281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc4d931061d4aaba3dc577413f06866', 'request_id': 'req-0f36b017-ea89-42cb-82fb-2988b77483a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.027320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0620f77ac274727b9995b261106e917', 'request_id': 'req-72a935e3-33b2-4709-b9b8-f5eef5e6b023'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.044447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0620f77ac274727b9995b261106e917', 'request_id': 'req-72a935e3-33b2-4709-b9b8-f5eef5e6b023'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.067548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] The token cache is being invalidate because project b0620f77ac274727b9995b261106e917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:24:45.067842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.069340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.077004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 669/1331] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:24:44 2022] DELETE /identity/v3/projects/b0620f77ac274727b9995b261106e917 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:24:45.078178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:45.082083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:45.087759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc4d931061d4aaba3dc577413f06866', 'request_id': 'req-0f36b017-ea89-42cb-82fb-2988b77483a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.108054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc4d931061d4aaba3dc577413f06866', 'request_id': 'req-0f36b017-ea89-42cb-82fb-2988b77483a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.133061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:45.141397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] The token cache is being invalidate because project 9fc4d931061d4aaba3dc577413f06866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:24:45.143495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.144562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f36b017-ea89-42cb-82fb-2988b77483a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.146965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 663/1332] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:24:44 2022] DELETE /identity/v3/projects/9fc4d931061d4aaba3dc577413f06866 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:45.157046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14767db-52fd-46cb-aa35-07f3e9b32190 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:45.157796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14767db-52fd-46cb-aa35-07f3e9b32190 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:45.158484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14767db-52fd-46cb-aa35-07f3e9b32190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:45.230338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a935e3-33b2-4709-b9b8-f5eef5e6b023 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:45.238274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f14767db-52fd-46cb-aa35-07f3e9b32190 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:45.243545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f14767db-52fd-46cb-aa35-07f3e9b32190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:45.251478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:45.252164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:45.252542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:45.252822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] PATH_INFO: `/v3/users/2092aa7cb2df4f9589a48675e538d3cd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:45.253986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2092aa7cb2df4f9589a48675e538d3cd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:45.255967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:45.271361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 664/1333] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:45.278516 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26784de0-bb6c-49bd-af15-ba82360dd7ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:45.279511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26784de0-bb6c-49bd-af15-ba82360dd7ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:45.281255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26784de0-bb6c-49bd-af15-ba82360dd7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:45.291805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:45.364573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-26784de0-bb6c-49bd-af15-ba82360dd7ad None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:45.367210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-26784de0-bb6c-49bd-af15-ba82360dd7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:45.392746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 665/1334] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:45.394968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2092aa7cb2df4f9589a48675e538d3cd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.395708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2092aa7cb2df4f9589a48675e538d3cd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.395998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26784de0-bb6c-49bd-af15-ba82360dd7ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:45.418166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26784de0-bb6c-49bd-af15-ba82360dd7ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:45.431310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:45.431899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:45.432202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:45.432463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:45.433135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:45.433438 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:45.435082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:45.445497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2092aa7cb2df4f9589a48675e538d3cd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.466957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2092aa7cb2df4f9589a48675e538d3cd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.467266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 666/1335] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:45.468114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2092aa7cb2df4f9589a48675e538d3cd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:45.470944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:45.472894 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:45.497203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 670/1336] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:24:45 2022] DELETE /identity/v3/users/2092aa7cb2df4f9589a48675e538d3cd => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:45.505759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b25a5-fed5-47ff-a8b0-908e011c32ee admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:45.651992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b38ec-8c59-46ae-86ee-b954b27e3b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:45.652671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b38ec-8c59-46ae-86ee-b954b27e3b7a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:45.653008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b38ec-8c59-46ae-86ee-b954b27e3b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:45.653901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b38ec-8c59-46ae-86ee-b954b27e3b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:45.654062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b38ec-8c59-46ae-86ee-b954b27e3b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:45.656407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b38ec-8c59-46ae-86ee-b954b27e3b7a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:45.745170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 667/1337] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:45 2022] POST /identity/v3/projects => generated 381 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:45.749308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:45.772108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98815357-136b-42ff-88cb-58b0b7902357 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:45.926516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c6ee1-21c0-4acd-9fca-eb87f66ccf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:45.927157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c6ee1-21c0-4acd-9fca-eb87f66ccf58 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:45.927440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c6ee1-21c0-4acd-9fca-eb87f66ccf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:45.927718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c6ee1-21c0-4acd-9fca-eb87f66ccf58 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:45.928364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c6ee1-21c0-4acd-9fca-eb87f66ccf58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:45.934642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c6ee1-21c0-4acd-9fca-eb87f66ccf58 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:45.987580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 671/1338] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:45 2022] POST /identity/v3/users => generated 291 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:45.992393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b38ec-8c59-46ae-86ee-b954b27e3b7a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:46.010191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b38ec-8c59-46ae-86ee-b954b27e3b7a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:46.020089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001a7c1-fb61-4a9e-967b-f847d2320466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:46.021126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001a7c1-fb61-4a9e-967b-f847d2320466 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.021510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001a7c1-fb61-4a9e-967b-f847d2320466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.021880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001a7c1-fb61-4a9e-967b-f847d2320466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.023895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001a7c1-fb61-4a9e-967b-f847d2320466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:46.028475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001a7c1-fb61-4a9e-967b-f847d2320466 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:46.054343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 668/1339] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:46.057834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c6ee1-21c0-4acd-9fca-eb87f66ccf58 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:46.068437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c6ee1-21c0-4acd-9fca-eb87f66ccf58 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:46.084637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4c2e7-c49f-41d1-ac54-58151b628927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:46.085399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c2e7-c49f-41d1-ac54-58151b628927 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.085766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c2e7-c49f-41d1-ac54-58151b628927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.086107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c2e7-c49f-41d1-ac54-58151b628927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.086997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4c2e7-c49f-41d1-ac54-58151b628927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:46.088777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4c2e7-c49f-41d1-ac54-58151b628927 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:46.116660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 672/1340] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:46.120240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001a7c1-fb61-4a9e-967b-f847d2320466 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:46.132970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001a7c1-fb61-4a9e-967b-f847d2320466 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:46.152777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31301467-7f44-4222-bac9-8ba406fa853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:46.154837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301467-7f44-4222-bac9-8ba406fa853e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.155308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301467-7f44-4222-bac9-8ba406fa853e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.157913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31301467-7f44-4222-bac9-8ba406fa853e admin admin] PATH_INFO: `/v3/projects/0dc60cbdb9f34257b4b29684cc3584ca/users/aa808ff135f6439e8c18929da619e494/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.194272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31301467-7f44-4222-bac9-8ba406fa853e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc60cbdb9f34257b4b29684cc3584ca, user_id=aa808ff135f6439e8c18929da619e494, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:46.202225 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31301467-7f44-4222-bac9-8ba406fa853e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:46.241406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 669/1341] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:46 2022] PUT /identity/v3/projects/0dc60cbdb9f34257b4b29684cc3584ca/users/aa808ff135f6439e8c18929da619e494/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:46.243152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822cde5-7ad0-4546-9794-7acbd88ce485 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.243552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822cde5-7ad0-4546-9794-7acbd88ce485 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.246670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822cde5-7ad0-4546-9794-7acbd88ce485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.265733 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b833d3e-c521-4967-9fd5-44c6a808dc9c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.266141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b833d3e-c521-4967-9fd5-44c6a808dc9c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.266621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b833d3e-c521-4967-9fd5-44c6a808dc9c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.269644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 670/1342] 10.222.0.16 () {54 vars in 912 bytes} [Mon Oct 17 10:24:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:24:46.278168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da1d8-5a5e-4a42-88fc-afa23d47c2af None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.279157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da1d8-5a5e-4a42-88fc-afa23d47c2af None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.279549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da1d8-5a5e-4a42-88fc-afa23d47c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.342766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d822cde5-7ad0-4546-9794-7acbd88ce485 None None] MFA Rules not processed for user `aa808ff135f6439e8c18929da619e494`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:46.350654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d822cde5-7ad0-4546-9794-7acbd88ce485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:46.380581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-633da1d8-5a5e-4a42-88fc-afa23d47c2af None None] MFA Rules not processed for user `bd7883e3f6ab4f58969f51ffd439b1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:46.384036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-633da1d8-5a5e-4a42-88fc-afa23d47c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:46.471117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:46.515497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 671/1343] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:24:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:46.516457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 673/1344] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:46.520911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da1d8-5a5e-4a42-88fc-afa23d47c2af None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:46.572821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:46.581912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822cde5-7ad0-4546-9794-7acbd88ce485 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:46.583344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d822cde5-7ad0-4546-9794-7acbd88ce485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:46.613291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822cde5-7ad0-4546-9794-7acbd88ce485 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:46.627851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd770b-c576-4b80-a2a8-5da117829d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:46.632793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd770b-c576-4b80-a2a8-5da117829d11 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.633102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd770b-c576-4b80-a2a8-5da117829d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.633593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd770b-c576-4b80-a2a8-5da117829d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.661848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd770b-c576-4b80-a2a8-5da117829d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:46.664195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd770b-c576-4b80-a2a8-5da117829d11 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:46.685020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 674/1345] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:46.707615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da1d8-5a5e-4a42-88fc-afa23d47c2af None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:46.717149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7af92-7d8b-40bc-8276-c9ed61e85cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:46.717775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7af92-7d8b-40bc-8276-c9ed61e85cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.718053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7af92-7d8b-40bc-8276-c9ed61e85cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.718342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7af92-7d8b-40bc-8276-c9ed61e85cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.719006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7af92-7d8b-40bc-8276-c9ed61e85cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:46.722455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7af92-7d8b-40bc-8276-c9ed61e85cc8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:46.773319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd770b-c576-4b80-a2a8-5da117829d11 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:46.775941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 672/1346] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:46 2022] POST /identity/v3/projects => generated 381 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:24:46.792970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd770b-c576-4b80-a2a8-5da117829d11 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:46.936410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116467b-1372-4902-a508-3dceaa545116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:46.937057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116467b-1372-4902-a508-3dceaa545116 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:46.937349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116467b-1372-4902-a508-3dceaa545116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:46.937605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116467b-1372-4902-a508-3dceaa545116 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:46.938244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116467b-1372-4902-a508-3dceaa545116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:46.940404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116467b-1372-4902-a508-3dceaa545116 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:46.995508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 675/1347] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:46 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:24:46.999107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7af92-7d8b-40bc-8276-c9ed61e85cc8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:47.051332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7af92-7d8b-40bc-8276-c9ed61e85cc8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:47.059760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73434c9b-0e88-4594-82aa-bbc9cd878a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:47.061107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434c9b-0e88-4594-82aa-bbc9cd878a59 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:47.061614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434c9b-0e88-4594-82aa-bbc9cd878a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:47.062098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434c9b-0e88-4594-82aa-bbc9cd878a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:47.063683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73434c9b-0e88-4594-82aa-bbc9cd878a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:47.065565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73434c9b-0e88-4594-82aa-bbc9cd878a59 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:47.094238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116467b-1372-4902-a508-3dceaa545116 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:47.095520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 673/1348] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:47.108870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116467b-1372-4902-a508-3dceaa545116 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:47.123597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223d0ea-bca6-4c58-b80b-93fe2c02705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:47.124303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223d0ea-bca6-4c58-b80b-93fe2c02705c admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:47.124778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223d0ea-bca6-4c58-b80b-93fe2c02705c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:47.125210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223d0ea-bca6-4c58-b80b-93fe2c02705c admin admin] PATH_INFO: `/v3/projects/45e9984c466947e3a537cde94dd36c62/users/c4299b7ca8fb4be2a3974463ed36a4dc/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:47.176042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223d0ea-bca6-4c58-b80b-93fe2c02705c admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e9984c466947e3a537cde94dd36c62, user_id=c4299b7ca8fb4be2a3974463ed36a4dc, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:47.178846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223d0ea-bca6-4c58-b80b-93fe2c02705c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:47.206954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 676/1349] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:47 2022] PUT /identity/v3/projects/45e9984c466947e3a537cde94dd36c62/users/c4299b7ca8fb4be2a3974463ed36a4dc/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:47.210731 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73434c9b-0e88-4594-82aa-bbc9cd878a59 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:47.235418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73434c9b-0e88-4594-82aa-bbc9cd878a59 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:47.387082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b01675-ec6e-446b-96af-f6e92b03e43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:47.387694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01675-ec6e-446b-96af-f6e92b03e43b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:47.387974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01675-ec6e-446b-96af-f6e92b03e43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:47.388245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01675-ec6e-446b-96af-f6e92b03e43b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:47.388961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b01675-ec6e-446b-96af-f6e92b03e43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:47.394642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b01675-ec6e-446b-96af-f6e92b03e43b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:47.417718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 674/1350] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:47.421735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223d0ea-bca6-4c58-b80b-93fe2c02705c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:47.437848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223d0ea-bca6-4c58-b80b-93fe2c02705c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:47.454468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:47.455440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:47.455910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:47.456395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] PATH_INFO: `/v3/projects/45e9984c466947e3a537cde94dd36c62/users/c4299b7ca8fb4be2a3974463ed36a4dc/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:47.463850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e9984c466947e3a537cde94dd36c62, user_id=c4299b7ca8fb4be2a3974463ed36a4dc, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:47.466966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:47.488720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 677/1351] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:47 2022] PUT /identity/v3/projects/45e9984c466947e3a537cde94dd36c62/users/c4299b7ca8fb4be2a3974463ed36a4dc/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:47.499716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d2336-d94e-4524-bad7-e4e9d0c18bd3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:47.499992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d2336-d94e-4524-bad7-e4e9d0c18bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:47.500357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d2336-d94e-4524-bad7-e4e9d0c18bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:47.541672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-214d2336-d94e-4524-bad7-e4e9d0c18bd3 None None] MFA Rules not processed for user `c4299b7ca8fb4be2a3974463ed36a4dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:47.543874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-214d2336-d94e-4524-bad7-e4e9d0c18bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:47.751226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 675/1352] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:24:47.764516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:47.766489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:47.767787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d2336-d94e-4524-bad7-e4e9d0c18bd3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:47.770566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-214d2336-d94e-4524-bad7-e4e9d0c18bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:47.849533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:47.896814 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fd33f-d096-4cee-98e6-3ba38c69b824 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:47.908126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61b587-fa5e-4030-8cdd-276cb0326052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:47.908976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61b587-fa5e-4030-8cdd-276cb0326052 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:47.909291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61b587-fa5e-4030-8cdd-276cb0326052 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:47.909636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61b587-fa5e-4030-8cdd-276cb0326052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:47.938559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61b587-fa5e-4030-8cdd-276cb0326052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:47.939927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61b587-fa5e-4030-8cdd-276cb0326052 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:47.967550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 678/1353] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:47.979803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d2336-d94e-4524-bad7-e4e9d0c18bd3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:48.001618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:48.002441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:48.002586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:48.003627 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:48.013100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:48.043300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:48.045286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:48.068775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 676/1354] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:24:48.123272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61b587-fa5e-4030-8cdd-276cb0326052 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:48.136261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61b587-fa5e-4030-8cdd-276cb0326052 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:48.163414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:48.163414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:48.175023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba9af8-0c4e-42d8-9276-7a6b6ff05b00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:48.175641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba9af8-0c4e-42d8-9276-7a6b6ff05b00 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:48.178806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba9af8-0c4e-42d8-9276-7a6b6ff05b00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:48.179379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba9af8-0c4e-42d8-9276-7a6b6ff05b00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:48.290313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:48.382378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba9af8-0c4e-42d8-9276-7a6b6ff05b00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:48.390086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba9af8-0c4e-42d8-9276-7a6b6ff05b00 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:48.409738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816f27-cb6d-46b0-9087-69dfe7ffd796 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:48.426040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:48.428162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:48.428446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:48.428724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] PATH_INFO: `/v3/projects/00fff1386985436b8addad058376ff66` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:48.434166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] RBAC: Authorizing `identity:delete_project(project_id=00fff1386985436b8addad058376ff66)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:48.436571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:48.442799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 679/1355] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:48.444520 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb9be-ebb3-482a-bd3d-b651ed713800 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:48.448630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb9be-ebb3-482a-bd3d-b651ed713800 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:48.450460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb9be-ebb3-482a-bd3d-b651ed713800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:48.517220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bf3eb9be-ebb3-482a-bd3d-b651ed713800 None None] MFA Rules not processed for user `aaedb0e1920a42eb97e8dc2f3ec76905`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:48.520260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bf3eb9be-ebb3-482a-bd3d-b651ed713800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:48.667510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00fff1386985436b8addad058376ff66', 'request_id': 'req-9af41559-462e-4579-a016-8e3630b9774c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:48.694652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 680/1356] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:48.715552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00fff1386985436b8addad058376ff66', 'request_id': 'req-9af41559-462e-4579-a016-8e3630b9774c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:48.740888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00fff1386985436b8addad058376ff66', 'request_id': 'req-9af41559-462e-4579-a016-8e3630b9774c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:48.806730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] The token cache is being invalidate because project 00fff1386985436b8addad058376ff66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:24:48.807105 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:48.812709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9af41559-462e-4579-a016-8e3630b9774c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:48.827652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 677/1357] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:24:48 2022] DELETE /identity/v3/projects/00fff1386985436b8addad058376ff66 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:24:48.832807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81903-3aff-4b1d-b5a2-19d7f5030d60 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:48.833368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81903-3aff-4b1d-b5a2-19d7f5030d60 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:48.833896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81903-3aff-4b1d-b5a2-19d7f5030d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:48.945708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b8f81903-3aff-4b1d-b5a2-19d7f5030d60 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:48.972458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b8f81903-3aff-4b1d-b5a2-19d7f5030d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:49.228296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 681/1358] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:49.237533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04dd32-338b-427c-9568-6a35ae534059 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:49.238878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04dd32-338b-427c-9568-6a35ae534059 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:49.239346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04dd32-338b-427c-9568-6a35ae534059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:49.293275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5d04dd32-338b-427c-9568-6a35ae534059 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:49.297877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5d04dd32-338b-427c-9568-6a35ae534059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:49.322987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 678/1359] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:49.323351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f81903-3aff-4b1d-b5a2-19d7f5030d60 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:49.341307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f81903-3aff-4b1d-b5a2-19d7f5030d60 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:49.355558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:49.357018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:49.359022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:49.361495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:49.362479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:49.377164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:49.379761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:49.436370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 682/1360] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:49.439024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04dd32-338b-427c-9568-6a35ae534059 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:49.454267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04dd32-338b-427c-9568-6a35ae534059 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:49.474810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675887f5-a0dd-4937-9244-623a5fa4711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:49.475379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675887f5-a0dd-4937-9244-623a5fa4711d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:49.475611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675887f5-a0dd-4937-9244-623a5fa4711d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:49.475881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675887f5-a0dd-4937-9244-623a5fa4711d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:49.481923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675887f5-a0dd-4937-9244-623a5fa4711d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:49.490517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675887f5-a0dd-4937-9244-623a5fa4711d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:49.581090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 679/1361] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:49 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:49.581969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:49.598976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a000406-af2b-4123-afaa-eefdac387ac0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:49.762767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467c536-8947-4983-879f-6ff9dc625247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:49.764187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467c536-8947-4983-879f-6ff9dc625247 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:49.764475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467c536-8947-4983-879f-6ff9dc625247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:49.764757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467c536-8947-4983-879f-6ff9dc625247 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:49.767531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467c536-8947-4983-879f-6ff9dc625247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:49.772208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467c536-8947-4983-879f-6ff9dc625247 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:49.809001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:49.841273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 683/1362] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:49 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:24:49.845525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675887f5-a0dd-4937-9244-623a5fa4711d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:49.865012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675887f5-a0dd-4937-9244-623a5fa4711d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:49.872318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735acb93-f2a1-4d7f-9020-8234a2219906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:49.873009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735acb93-f2a1-4d7f-9020-8234a2219906 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:49.873362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735acb93-f2a1-4d7f-9020-8234a2219906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:49.873662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735acb93-f2a1-4d7f-9020-8234a2219906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:49.874463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735acb93-f2a1-4d7f-9020-8234a2219906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:49.876196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735acb93-f2a1-4d7f-9020-8234a2219906 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:49.878273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:49.906998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 680/1363] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:49.908214 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467c536-8947-4983-879f-6ff9dc625247 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:49.934273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467c536-8947-4983-879f-6ff9dc625247 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:49.963755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a2636-752a-42a6-9bc2-86a23a84416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:49.964520 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2636-752a-42a6-9bc2-86a23a84416b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:49.964739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2636-752a-42a6-9bc2-86a23a84416b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:49.965907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2636-752a-42a6-9bc2-86a23a84416b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:49.965907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a2636-752a-42a6-9bc2-86a23a84416b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:49.971396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a2636-752a-42a6-9bc2-86a23a84416b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:50.021689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 684/1364] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:50.023243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735acb93-f2a1-4d7f-9020-8234a2219906 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:50.036004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735acb93-f2a1-4d7f-9020-8234a2219906 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:50.053332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c919f2-7647-4950-8a03-f758e34c29a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:50.054739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c919f2-7647-4950-8a03-f758e34c29a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:50.056715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c919f2-7647-4950-8a03-f758e34c29a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:50.057241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c919f2-7647-4950-8a03-f758e34c29a7 admin admin] PATH_INFO: `/v3/projects/372090d2809443cd8db49639092a1eb5/users/8921b0f17eee4919bef2fc6906a18048/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:50.152913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c919f2-7647-4950-8a03-f758e34c29a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372090d2809443cd8db49639092a1eb5, user_id=8921b0f17eee4919bef2fc6906a18048, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:50.157558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c919f2-7647-4950-8a03-f758e34c29a7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:50.196039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 681/1365] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:50 2022] PUT /identity/v3/projects/372090d2809443cd8db49639092a1eb5/users/8921b0f17eee4919bef2fc6906a18048/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:24:50.207124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faebdea-257e-464c-b917-8d4efd788ee7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:50.208128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faebdea-257e-464c-b917-8d4efd788ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:50.208410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faebdea-257e-464c-b917-8d4efd788ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:50.339840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9faebdea-257e-464c-b917-8d4efd788ee7 None None] MFA Rules not processed for user `8921b0f17eee4919bef2fc6906a18048`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:50.341081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9faebdea-257e-464c-b917-8d4efd788ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:50.454680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5640a67-a7a2-45f3-bd27-c8752514a10a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:50.455572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5640a67-a7a2-45f3-bd27-c8752514a10a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:50.456067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5640a67-a7a2-45f3-bd27-c8752514a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:50.572984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e5640a67-a7a2-45f3-bd27-c8752514a10a None None] MFA Rules not processed for user `ca96437c5fdc4c1ba41ca7960813dfae`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:50.577151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e5640a67-a7a2-45f3-bd27-c8752514a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:50.690085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 685/1366] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:24:50.692830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faebdea-257e-464c-b917-8d4efd788ee7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:50.866292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 682/1367] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:50.868481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c175611-3dcf-4876-bad2-0e1932e36806 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:50.869722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c175611-3dcf-4876-bad2-0e1932e36806 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:50.871174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c175611-3dcf-4876-bad2-0e1932e36806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:50.904386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faebdea-257e-464c-b917-8d4efd788ee7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:50.940305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:50.943210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:50.943210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:50.943210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:50.943210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:50.944457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:50.954840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4c175611-3dcf-4876-bad2-0e1932e36806 None None] MFA Rules not processed for user `ca96437c5fdc4c1ba41ca7960813dfae`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:50.956307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4c175611-3dcf-4876-bad2-0e1932e36806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:51.013080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 683/1368] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:51.028764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7518-971d-4d40-baee-3a52037e2f8f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:51.029047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7518-971d-4d40-baee-3a52037e2f8f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:51.029320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7518-971d-4d40-baee-3a52037e2f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:51.033898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 686/1369] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:50 2022] POST /identity/v3/projects => generated 381 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:51.033898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:51.035564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:51.188268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-55df7518-971d-4d40-baee-3a52037e2f8f None None] MFA Rules not processed for user `6b4b2700c20946ea93a1d20d7dea929f`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:51.197718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-55df7518-971d-4d40-baee-3a52037e2f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:51.208320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:51.272896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e552bae-b0ea-4c67-9eda-cfe1ccdbada5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:51.282226 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:51.283229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:51.299714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:51.300306 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:51.427439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 684/1370] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:51.428619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df7518-971d-4d40-baee-3a52037e2f8f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:51.490752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:51.492766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:51.539838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 687/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:51.571255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:51.581148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:51.582820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:51.618018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabfced-1761-4b44-8d51-a641563b3ad8 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:51.630939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df7518-971d-4d40-baee-3a52037e2f8f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:51.646780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325412b-0a7b-44f0-a540-fc26b5ba706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:51.647413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325412b-0a7b-44f0-a540-fc26b5ba706d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:51.647695 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325412b-0a7b-44f0-a540-fc26b5ba706d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:51.647862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eb91e-ec9c-44e5-b4f9-f4740d995d29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:51.648026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325412b-0a7b-44f0-a540-fc26b5ba706d admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:51.648647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325412b-0a7b-44f0-a540-fc26b5ba706d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:51.649184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eb91e-ec9c-44e5-b4f9-f4740d995d29 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:51.649803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eb91e-ec9c-44e5-b4f9-f4740d995d29 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:51.650235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eb91e-ec9c-44e5-b4f9-f4740d995d29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:51.650834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325412b-0a7b-44f0-a540-fc26b5ba706d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:51.721953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 685/1372] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:51 2022] POST /identity/v3/users => generated 297 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:51.722387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eb91e-ec9c-44e5-b4f9-f4740d995d29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:51.765032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325412b-0a7b-44f0-a540-fc26b5ba706d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:51.765307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eb91e-ec9c-44e5-b4f9-f4740d995d29 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:51.765553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325412b-0a7b-44f0-a540-fc26b5ba706d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:51.772465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 688/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:24:51.783681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267507e-607a-4002-9547-77239a8cdb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:51.783918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267507e-607a-4002-9547-77239a8cdb25 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:51.784074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267507e-607a-4002-9547-77239a8cdb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:51.785197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267507e-607a-4002-9547-77239a8cdb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:51.785197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267507e-607a-4002-9547-77239a8cdb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:51.786809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267507e-607a-4002-9547-77239a8cdb25 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:51.832282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eb91e-ec9c-44e5-b4f9-f4740d995d29 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:51.832616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 686/1374] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:51.871535 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eb91e-ec9c-44e5-b4f9-f4740d995d29 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:51.899293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fae85-d518-4d8b-97d2-30a53226cc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:51.900900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fae85-d518-4d8b-97d2-30a53226cc57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:51.901373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fae85-d518-4d8b-97d2-30a53226cc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:51.902687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fae85-d518-4d8b-97d2-30a53226cc57 admin admin] PATH_INFO: `/v3/projects/bf56dfd9e30c476bba8b1e11b7e0f336/users/c6e41c71c8964d069031579467dfed60/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:51.977987 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fae85-d518-4d8b-97d2-30a53226cc57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf56dfd9e30c476bba8b1e11b7e0f336, user_id=c6e41c71c8964d069031579467dfed60, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:51.983630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fae85-d518-4d8b-97d2-30a53226cc57 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:52.019737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 689/1375] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:51 2022] PUT /identity/v3/projects/bf56dfd9e30c476bba8b1e11b7e0f336/users/c6e41c71c8964d069031579467dfed60/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:24:52.023520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267507e-607a-4002-9547-77239a8cdb25 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:52.034064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267507e-607a-4002-9547-77239a8cdb25 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:52.208599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196035ec-f644-4b8e-8313-f4f1d85e7aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:52.215969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196035ec-f644-4b8e-8313-f4f1d85e7aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:52.216593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196035ec-f644-4b8e-8313-f4f1d85e7aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:52.217100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196035ec-f644-4b8e-8313-f4f1d85e7aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:52.223085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196035ec-f644-4b8e-8313-f4f1d85e7aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:52.225126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196035ec-f644-4b8e-8313-f4f1d85e7aa4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:52.285629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 687/1376] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:24:52.286874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fae85-d518-4d8b-97d2-30a53226cc57 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:52.305932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fae85-d518-4d8b-97d2-30a53226cc57 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:52.318666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:52.319724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:52.320228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:52.323257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] PATH_INFO: `/v3/projects/bf56dfd9e30c476bba8b1e11b7e0f336/users/c6e41c71c8964d069031579467dfed60/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:52.329108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf56dfd9e30c476bba8b1e11b7e0f336, user_id=c6e41c71c8964d069031579467dfed60, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:52.333161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:52.408395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 690/1377] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:52 2022] PUT /identity/v3/projects/bf56dfd9e30c476bba8b1e11b7e0f336/users/c6e41c71c8964d069031579467dfed60/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:24:52.420177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de824264-5d32-4337-bf88-184d59538b2c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:52.420722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de824264-5d32-4337-bf88-184d59538b2c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:52.421193 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de824264-5d32-4337-bf88-184d59538b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:52.501927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-de824264-5d32-4337-bf88-184d59538b2c None None] MFA Rules not processed for user `c6e41c71c8964d069031579467dfed60`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:52.503603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-de824264-5d32-4337-bf88-184d59538b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:52.521663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:52.524671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:52.620180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 688/1378] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:52.635332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de824264-5d32-4337-bf88-184d59538b2c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:52.639460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-de824264-5d32-4337-bf88-184d59538b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:52.723406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0847f-5e0c-4c10-aaf0-10dbda9d716a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:52.739392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:52.742867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:52.746097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:52.746429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] PATH_INFO: `/v3/users/7bc995383f9e47f5903ddf0c260c01fd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:52.749736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bc995383f9e47f5903ddf0c260c01fd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:52.754826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:52.779885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:52.829231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de824264-5d32-4337-bf88-184d59538b2c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:52.843998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b0872-ca17-40cc-8c9d-b23a7e1eaa18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:52.847335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0872-ca17-40cc-8c9d-b23a7e1eaa18 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:52.853665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0872-ca17-40cc-8c9d-b23a7e1eaa18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:52.854730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0872-ca17-40cc-8c9d-b23a7e1eaa18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:52.923453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0872-ca17-40cc-8c9d-b23a7e1eaa18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:52.925213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0872-ca17-40cc-8c9d-b23a7e1eaa18 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:52.937862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:52.940897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bc995383f9e47f5903ddf0c260c01fd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:52.942019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bc995383f9e47f5903ddf0c260c01fd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:52.948859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 689/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:52.955044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67380595-699f-4f8d-933f-1e7b99c903e9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:52.955329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67380595-699f-4f8d-933f-1e7b99c903e9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:52.955605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67380595-699f-4f8d-933f-1e7b99c903e9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:52.959404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 690/1380] 10.222.0.16 () {54 vars in 912 bytes} [Mon Oct 17 10:24:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:24:52.965606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ad74c-f9e1-436f-a4ec-1f4e181dacb7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:52.965891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ad74c-f9e1-436f-a4ec-1f4e181dacb7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:52.966168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ad74c-f9e1-436f-a4ec-1f4e181dacb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:52.969770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bc995383f9e47f5903ddf0c260c01fd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:52.983858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bc995383f9e47f5903ddf0c260c01fd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:52.985450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-14d6ff53-b5fe-4d47-b794-e254787fd009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bc995383f9e47f5903ddf0c260c01fd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:53.030006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 691/1381] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:24:52 2022] DELETE /identity/v3/users/7bc995383f9e47f5903ddf0c260c01fd => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:53.036614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563a302-b0fd-4c0f-9d72-0134bb699b69 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:53.037010 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563a302-b0fd-4c0f-9d72-0134bb699b69 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:53.037394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563a302-b0fd-4c0f-9d72-0134bb699b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:53.096918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4563a302-b0fd-4c0f-9d72-0134bb699b69 None None] MFA Rules not processed for user `bd7883e3f6ab4f58969f51ffd439b1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:53.099374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4563a302-b0fd-4c0f-9d72-0134bb699b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:53.137488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f00ad74c-f9e1-436f-a4ec-1f4e181dacb7 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:53.141556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f00ad74c-f9e1-436f-a4ec-1f4e181dacb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:53.284099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 692/1382] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:53.285982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfbee3-8055-41d7-b23e-4fee7017314d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:53.286567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfbee3-8055-41d7-b23e-4fee7017314d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:53.286841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfbee3-8055-41d7-b23e-4fee7017314d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:53.345878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 691/1383] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:53.368435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a9bfbee3-8055-41d7-b23e-4fee7017314d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:53.370671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a9bfbee3-8055-41d7-b23e-4fee7017314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:53.606366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ad74c-f9e1-436f-a4ec-1f4e181dacb7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:53.607954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f00ad74c-f9e1-436f-a4ec-1f4e181dacb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:53.624650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 693/1384] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:53.693901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfbee3-8055-41d7-b23e-4fee7017314d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:53.695406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a9bfbee3-8055-41d7-b23e-4fee7017314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:53.735033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfbee3-8055-41d7-b23e-4fee7017314d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:53.745517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:53.746142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:53.746511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:53.746717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:53.769120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:53.770820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:53.807063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 694/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:24:53.808251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:53.816062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:53.846611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1b25-498f-400d-9458-a4ff90039a68 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:53.853356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:53.854243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:53.856121 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:53.856918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:53.886862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:53.888576 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:53.902826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 695/1386] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:53.903790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:53.905753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:53.906158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ad74c-f9e1-436f-a4ec-1f4e181dacb7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:53.916208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaf539-9e04-42b9-be2c-a2b7b6b513c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:53.916837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf539-9e04-42b9-be2c-a2b7b6b513c9 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:53.917113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf539-9e04-42b9-be2c-a2b7b6b513c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:53.917382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf539-9e04-42b9-be2c-a2b7b6b513c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:53.929532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaf539-9e04-42b9-be2c-a2b7b6b513c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:53.932488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaf539-9e04-42b9-be2c-a2b7b6b513c9 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:53.941638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 692/1387] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:24:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:24:53.942912 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133f74b-774a-4452-8142-6b3282624bfd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:53.943350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133f74b-774a-4452-8142-6b3282624bfd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:53.943555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133f74b-774a-4452-8142-6b3282624bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:53.990840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:54.006028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8133f74b-774a-4452-8142-6b3282624bfd None None] MFA Rules not processed for user `eb7c6da9fd3c4b72a687daa923d07918`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:54.008707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8133f74b-774a-4452-8142-6b3282624bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:54.030428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:54.043624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d503c-0f65-4f13-9ed3-4d402381a609 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:54.049987 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e41c71c8964d069031579467dfed60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf56dfd9e30c476bba8b1e11b7e0f336', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:54.051233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:54.051702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:54.052318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:54.053269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:54.053751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:54.057016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:54.087705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 696/1388] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:54.089734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:54.113035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424bcb4-c4d1-4391-ba52-cf048ef350a2 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:54.120708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25b850-748d-4eff-ab96-5ee5ba9f8ff8 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e41c71c8964d069031579467dfed60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf56dfd9e30c476bba8b1e11b7e0f336', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:54.121825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25b850-748d-4eff-ab96-5ee5ba9f8ff8 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:54.123170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25b850-748d-4eff-ab96-5ee5ba9f8ff8 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:54.123934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25b850-748d-4eff-ab96-5ee5ba9f8ff8 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:54.125457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25b850-748d-4eff-ab96-5ee5ba9f8ff8 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:54.132126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25b850-748d-4eff-ab96-5ee5ba9f8ff8 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:54.174913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 693/1389] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:54.191137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133f74b-774a-4452-8142-6b3282624bfd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:54.195910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8133f74b-774a-4452-8142-6b3282624bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:54.197116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 697/1390] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:54 2022] POST /identity/v3/projects => generated 355 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:54.198296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25b850-748d-4eff-ab96-5ee5ba9f8ff8 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:54.392232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25b850-748d-4eff-ab96-5ee5ba9f8ff8 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:54.420204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e41c71c8964d069031579467dfed60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf56dfd9e30c476bba8b1e11b7e0f336', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:54.420664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133f74b-774a-4452-8142-6b3282624bfd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:54.421967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:54.423037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:54.425827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:54.429297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:54.429297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:54.435908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:54.479049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 698/1391] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:54.611641 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:54.623293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:54.623638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:54.623983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:54.624263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:54.643170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec09d2-fde7-4d0a-bc8c-3c5501c22d7e tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:54.646018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e41c71c8964d069031579467dfed60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf56dfd9e30c476bba8b1e11b7e0f336', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:54.646714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:54.647145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:54.647425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] PATH_INFO: `/v3/projects/ea053653c1e34961bb9068095a5a49ea` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:54.649783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ea053653c1e34961bb9068095a5a49ea)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:54.652489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:54.764072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea053653c1e34961bb9068095a5a49ea', 'request_id': 'req-bcd5894a-e947-4fa7-8538-374d3a8ffd54'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:54.777758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:54.779899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea053653c1e34961bb9068095a5a49ea', 'request_id': 'req-bcd5894a-e947-4fa7-8538-374d3a8ffd54'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:54.788012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:54.788905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:54.817205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea053653c1e34961bb9068095a5a49ea', 'request_id': 'req-bcd5894a-e947-4fa7-8538-374d3a8ffd54'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:54.820585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 694/1392] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:54.873415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] The token cache is being invalidate because project ea053653c1e34961bb9068095a5a49ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:24:54.874439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:54.878487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bcd5894a-e947-4fa7-8538-374d3a8ffd54 tempest-VolumeQuotaClassesTest-1532175065 tempest-VolumeQuotaClassesTest-1532175065-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:54.885235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 699/1393] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:24:54 2022] DELETE /identity/v3/projects/ea053653c1e34961bb9068095a5a49ea => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:24:55.605651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:55.608426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:55.801311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3189f3-daaf-455e-8956-454dca979f19 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:55.909526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:55.911440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:55.912015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:55.912658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] PATH_INFO: `/v3/users/8921b0f17eee4919bef2fc6906a18048` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:55.914924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8921b0f17eee4919bef2fc6906a18048)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:55.917846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:55.999613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8921b0f17eee4919bef2fc6906a18048'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:56.000437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8921b0f17eee4919bef2fc6906a18048'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:56.016132 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8921b0f17eee4919bef2fc6906a18048'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:56.046938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8921b0f17eee4919bef2fc6906a18048'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:56.050445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8921b0f17eee4919bef2fc6906a18048'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:56.094123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 695/1394] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:24:55 2022] DELETE /identity/v3/users/8921b0f17eee4919bef2fc6906a18048 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:56.098710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd479f0-9e53-473b-a26b-3b0dc2abeb45 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:56.099615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd479f0-9e53-473b-a26b-3b0dc2abeb45 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:56.100177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd479f0-9e53-473b-a26b-3b0dc2abeb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:56.143567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bdd479f0-9e53-473b-a26b-3b0dc2abeb45 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:56.155203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bdd479f0-9e53-473b-a26b-3b0dc2abeb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:56.290858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:56.374410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 700/1395] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:56.392890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:56.395137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:56.451019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:56.543498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ead6a-bc25-4bef-a36f-52537d10eda8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:56.557820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada103b-2de8-4c50-82da-ea729d5d46db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:56.558729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada103b-2de8-4c50-82da-ea729d5d46db service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:56.559935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada103b-2de8-4c50-82da-ea729d5d46db service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:56.560368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada103b-2de8-4c50-82da-ea729d5d46db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:56.585189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada103b-2de8-4c50-82da-ea729d5d46db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:56.587088 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada103b-2de8-4c50-82da-ea729d5d46db service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:56.614936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 696/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:56.770462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd479f0-9e53-473b-a26b-3b0dc2abeb45 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:56.772061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bdd479f0-9e53-473b-a26b-3b0dc2abeb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:56.789879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd479f0-9e53-473b-a26b-3b0dc2abeb45 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:56.810172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:56.812555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:56.813920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:56.814036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] PATH_INFO: `/v3/projects/0841a46c829c45a29392017401b16ece` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:56.821403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0841a46c829c45a29392017401b16ece)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:56.841792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:57.033531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0841a46c829c45a29392017401b16ece', 'request_id': 'req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:57.058338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0841a46c829c45a29392017401b16ece', 'request_id': 'req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:57.104051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0841a46c829c45a29392017401b16ece', 'request_id': 'req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:57.136013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] The token cache is being invalidate because project 0841a46c829c45a29392017401b16ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:24:57.136338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:57.137370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-97a6b9f9-47d8-4ecb-b3ab-a7ba071277d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:57.143586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 701/1397] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:24:56 2022] DELETE /identity/v3/projects/0841a46c829c45a29392017401b16ece => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:57.149376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb9f0-0b0f-448a-b440-104a9041e066 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:57.149890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb9f0-0b0f-448a-b440-104a9041e066 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:57.150360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb9f0-0b0f-448a-b440-104a9041e066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:57.270921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-698cb9f0-0b0f-448a-b440-104a9041e066 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:57.272308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-698cb9f0-0b0f-448a-b440-104a9041e066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:57.292111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359172e-9732-4b2a-aa58-662544aaf41b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:57.292668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359172e-9732-4b2a-aa58-662544aaf41b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:57.294994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359172e-9732-4b2a-aa58-662544aaf41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:57.358131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b359172e-9732-4b2a-aa58-662544aaf41b None None] MFA Rules not processed for user `bd7883e3f6ab4f58969f51ffd439b1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:57.361735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b359172e-9732-4b2a-aa58-662544aaf41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:57.547732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 697/1398] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:24:57.557170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b6040-f911-4237-87cf-27c38cc7755e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:57.557690 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b6040-f911-4237-87cf-27c38cc7755e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:57.557972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b6040-f911-4237-87cf-27c38cc7755e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:57.645047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 702/1399] 10.222.0.30 () {58 vars in 1015 bytes} [Mon Oct 17 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:24:57.648749 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7e9b6040-f911-4237-87cf-27c38cc7755e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:57.649957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7e9b6040-f911-4237-87cf-27c38cc7755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:57.669307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 698/1400] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:24:57.673617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359172e-9732-4b2a-aa58-662544aaf41b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:57.692818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359172e-9732-4b2a-aa58-662544aaf41b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:57.704494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:57.705364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:57.705766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:57.706149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:57.707150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:24:57.707553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:57.711974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:57.728774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b6040-f911-4237-87cf-27c38cc7755e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:57.731841 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7e9b6040-f911-4237-87cf-27c38cc7755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:57.768631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b6040-f911-4237-87cf-27c38cc7755e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:57.768898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 703/1401] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:24:57.769777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:57.787026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24322a8d-a495-4923-944b-22c5482c7428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:57.787738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24322a8d-a495-4923-944b-22c5482c7428 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:57.788091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24322a8d-a495-4923-944b-22c5482c7428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:57.788451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24322a8d-a495-4923-944b-22c5482c7428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:57.801930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cbe5b-1e8f-4b9a-8c85-3d014da44e8f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:57.811908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24322a8d-a495-4923-944b-22c5482c7428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:57.818125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe0a8b-c285-4367-88e5-8ea3e285831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:57.818849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe0a8b-c285-4367-88e5-8ea3e285831c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:57.819129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe0a8b-c285-4367-88e5-8ea3e285831c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:57.826851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe0a8b-c285-4367-88e5-8ea3e285831c admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:57.826851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe0a8b-c285-4367-88e5-8ea3e285831c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:57.827466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24322a8d-a495-4923-944b-22c5482c7428 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:57.827547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe0a8b-c285-4367-88e5-8ea3e285831c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:57.866190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 699/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:24:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:24:57.904315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 704/1403] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:57 2022] POST /identity/v3/projects => generated 401 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:57.905430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24322a8d-a495-4923-944b-22c5482c7428 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:57.928559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24322a8d-a495-4923-944b-22c5482c7428 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:58.142027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147702d-d995-4af9-8002-23a763fa44ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:58.142920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147702d-d995-4af9-8002-23a763fa44ae admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:58.143768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147702d-d995-4af9-8002-23a763fa44ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:58.144658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147702d-d995-4af9-8002-23a763fa44ae admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:58.146266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147702d-d995-4af9-8002-23a763fa44ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:58.151291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147702d-d995-4af9-8002-23a763fa44ae admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:58.196003 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 700/1404] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:57 2022] POST /identity/v3/users => generated 301 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:24:58.200642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe0a8b-c285-4367-88e5-8ea3e285831c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:58.214740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe0a8b-c285-4367-88e5-8ea3e285831c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:58.227490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c35296-7e62-4458-a312-f9ef0c2b3c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:58.232378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c35296-7e62-4458-a312-f9ef0c2b3c6b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:58.233300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c35296-7e62-4458-a312-f9ef0c2b3c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:58.234489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c35296-7e62-4458-a312-f9ef0c2b3c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:58.235428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c35296-7e62-4458-a312-f9ef0c2b3c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:58.237519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c35296-7e62-4458-a312-f9ef0c2b3c6b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:58.252178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 705/1405] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:58.255847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147702d-d995-4af9-8002-23a763fa44ae admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:58.272533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147702d-d995-4af9-8002-23a763fa44ae admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:58.280394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:58.281346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:58.281836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:58.282290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:58.283271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:58.286368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:58.309350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 701/1406] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:24:58.311604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c35296-7e62-4458-a312-f9ef0c2b3c6b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:58.328917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c35296-7e62-4458-a312-f9ef0c2b3c6b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:58.342699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3b135-da83-4f7e-88bc-8c4d823afe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:58.343687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3b135-da83-4f7e-88bc-8c4d823afe9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:58.344239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3b135-da83-4f7e-88bc-8c4d823afe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:58.344751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3b135-da83-4f7e-88bc-8c4d823afe9b admin admin] PATH_INFO: `/v3/projects/750b570dc8cd4ca484d8d88727495826/users/bfce506069304a52932369a4d3770c95/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:58.381355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:58.384889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:58.432221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3b135-da83-4f7e-88bc-8c4d823afe9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=750b570dc8cd4ca484d8d88727495826, user_id=bfce506069304a52932369a4d3770c95, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:58.441584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3b135-da83-4f7e-88bc-8c4d823afe9b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:58.457966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5428f0a-6e8e-48db-b62a-f23d090768bb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:58.470165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:58.472270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:58.472270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:58.472270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] PATH_INFO: `/v3/projects/372090d2809443cd8db49639092a1eb5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:58.474591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=372090d2809443cd8db49639092a1eb5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:58.478673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:58.482654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:58.500415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 706/1407] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:58 2022] PUT /identity/v3/projects/750b570dc8cd4ca484d8d88727495826/users/bfce506069304a52932369a4d3770c95/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:58.512390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:58.514493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f35fe-d4a4-4b8b-9603-3b3932d32797 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:58.515361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f35fe-d4a4-4b8b-9603-3b3932d32797 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:58.517056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f35fe-d4a4-4b8b-9603-3b3932d32797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:58.657459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372090d2809443cd8db49639092a1eb5', 'request_id': 'req-82913837-f65b-4650-b057-02a8ff5334b2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:58.700548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-794f35fe-d4a4-4b8b-9603-3b3932d32797 None None] MFA Rules not processed for user `bfce506069304a52932369a4d3770c95`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:24:58.705791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-794f35fe-d4a4-4b8b-9603-3b3932d32797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:58.706226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372090d2809443cd8db49639092a1eb5', 'request_id': 'req-82913837-f65b-4650-b057-02a8ff5334b2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:58.756072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372090d2809443cd8db49639092a1eb5', 'request_id': 'req-82913837-f65b-4650-b057-02a8ff5334b2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:58.834787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] The token cache is being invalidate because project 372090d2809443cd8db49639092a1eb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:24:58.837200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:58.840525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:58.845785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 702/1408] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:24:58 2022] DELETE /identity/v3/projects/372090d2809443cd8db49639092a1eb5 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:58.853560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:58.865341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:58.948827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 707/1409] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:24:58.954940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f35fe-d4a4-4b8b-9603-3b3932d32797 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:58.957532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-794f35fe-d4a4-4b8b-9603-3b3932d32797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:59.173114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f35fe-d4a4-4b8b-9603-3b3932d32797 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:59.190543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:59.193741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:59.196651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:59.198506 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:59.202216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:59.209079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:59.210686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82913837-f65b-4650-b057-02a8ff5334b2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:59.227717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:59.228592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:59.231258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:59.232846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] PATH_INFO: `/v3/users/c6e41c71c8964d069031579467dfed60` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:59.237335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e41c71c8964d069031579467dfed60)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:59.243638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:59.323120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 708/1410] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:24:58 2022] POST /identity/v3/projects => generated 401 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:59.325521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:59.338181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:59.433485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e41c71c8964d069031579467dfed60'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:59.434205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e41c71c8964d069031579467dfed60'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:59.453269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e41c71c8964d069031579467dfed60'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:59.516784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e41c71c8964d069031579467dfed60'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:59.520554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e41c71c8964d069031579467dfed60'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:24:59.560738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 703/1411] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:24:58 2022] DELETE /identity/v3/users/c6e41c71c8964d069031579467dfed60 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:59.562916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:59.564628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:24:59.608559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:59.626235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac583bf-bec4-42df-a65c-2b3432abbb81 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:59.670221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde6e0e-bd09-47a7-9c14-f426e4b48ddb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:59.676296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ba90-65e9-4380-971b-83cc006a30cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:59.677304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ba90-65e9-4380-971b-83cc006a30cc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:59.677580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ba90-65e9-4380-971b-83cc006a30cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:59.678231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ba90-65e9-4380-971b-83cc006a30cc admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:59.680606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854ba90-65e9-4380-971b-83cc006a30cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:59.682886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854ba90-65e9-4380-971b-83cc006a30cc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:59.687017 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:24:59.723292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 704/1412] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:24:59 2022] POST /identity/v3/users => generated 307 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:24:59.724373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ba90-65e9-4380-971b-83cc006a30cc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:59.739485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ba90-65e9-4380-971b-83cc006a30cc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:59.746806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a9fc1-610b-4896-a1d8-d90e41e8f09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:59.747343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a9fc1-610b-4896-a1d8-d90e41e8f09f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:59.747712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a9fc1-610b-4896-a1d8-d90e41e8f09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:59.748007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a9fc1-610b-4896-a1d8-d90e41e8f09f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:59.749027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a9fc1-610b-4896-a1d8-d90e41e8f09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:59.751342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a9fc1-610b-4896-a1d8-d90e41e8f09f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:59.781106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 705/1413] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:24:59.782446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a9fc1-610b-4896-a1d8-d90e41e8f09f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:24:59.830730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a9fc1-610b-4896-a1d8-d90e41e8f09f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:24:59.856145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae59705-3035-4098-bfb0-7bb27d176b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:59.856932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae59705-3035-4098-bfb0-7bb27d176b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:59.857237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae59705-3035-4098-bfb0-7bb27d176b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:59.857407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:24:59.857566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae59705-3035-4098-bfb0-7bb27d176b9c admin admin] PATH_INFO: `/v3/projects/4b8d975b0a7a41ad81d8842b5d50aa69/users/19b2e84cf05d43c9a644dd8ff983826e/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:59.858240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:24:59.858865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:24:59.858986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] PATH_INFO: `/v3/users/5bddc8c303bd4c67bffbbeb68ead5f51` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:24:59.861537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bddc8c303bd4c67bffbbeb68ead5f51)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:59.864284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:59.923517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae59705-3035-4098-bfb0-7bb27d176b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8d975b0a7a41ad81d8842b5d50aa69, user_id=19b2e84cf05d43c9a644dd8ff983826e, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:24:59.937999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae59705-3035-4098-bfb0-7bb27d176b9c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:24:59.993283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 706/1414] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:24:59 2022] PUT /identity/v3/projects/4b8d975b0a7a41ad81d8842b5d50aa69/users/19b2e84cf05d43c9a644dd8ff983826e/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:24:59.998613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae59705-3035-4098-bfb0-7bb27d176b9c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:00.093816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bddc8c303bd4c67bffbbeb68ead5f51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:00.098223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bddc8c303bd4c67bffbbeb68ead5f51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:00.150689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bddc8c303bd4c67bffbbeb68ead5f51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:00.167581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bddc8c303bd4c67bffbbeb68ead5f51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:00.168206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3181ea3e-f7d8-4565-b221-ef671c7bf38d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bddc8c303bd4c67bffbbeb68ead5f51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:00.201713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 709/1415] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:24:59 2022] DELETE /identity/v3/users/5bddc8c303bd4c67bffbbeb68ead5f51 => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:00.205378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091f91-098a-4ee6-9ace-c928a827775c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:00.206024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091f91-098a-4ee6-9ace-c928a827775c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:00.206129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091f91-098a-4ee6-9ace-c928a827775c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:00.252579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae59705-3035-4098-bfb0-7bb27d176b9c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:00.288144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6e091f91-098a-4ee6-9ace-c928a827775c None None] MFA Rules not processed for user `eb7c6da9fd3c4b72a687daa923d07918`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:00.308904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6e091f91-098a-4ee6-9ace-c928a827775c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:00.577102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 710/1416] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:00.583559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670944e8-94d8-451b-bc5f-13b3bad8e7e9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:00.587010 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670944e8-94d8-451b-bc5f-13b3bad8e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:00.587347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670944e8-94d8-451b-bc5f-13b3bad8e7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:00.641320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-670944e8-94d8-451b-bc5f-13b3bad8e7e9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:00.642572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-670944e8-94d8-451b-bc5f-13b3bad8e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:00.671483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:00.672949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:00.673387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:00.675480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:00.676365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:00.682775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 711/1417] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:00.691464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:00.744998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 707/1418] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 747 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:00.752443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670944e8-94d8-451b-bc5f-13b3bad8e7e9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:00.754895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-670944e8-94d8-451b-bc5f-13b3bad8e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:00.767658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:00.777597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:00.791196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670944e8-94d8-451b-bc5f-13b3bad8e7e9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:00.827254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf30a6-7fb2-4f3c-9651-845e5cc57d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:00.827988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf30a6-7fb2-4f3c-9651-845e5cc57d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:00.828245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf30a6-7fb2-4f3c-9651-845e5cc57d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:00.828503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf30a6-7fb2-4f3c-9651-845e5cc57d30 admin admin] PATH_INFO: `/v3/projects/4b8d975b0a7a41ad81d8842b5d50aa69/users/19b2e84cf05d43c9a644dd8ff983826e/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:00.831265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf30a6-7fb2-4f3c-9651-845e5cc57d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8d975b0a7a41ad81d8842b5d50aa69, user_id=19b2e84cf05d43c9a644dd8ff983826e, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:00.834073 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf30a6-7fb2-4f3c-9651-845e5cc57d30 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:00.886109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:00.965024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 712/1419] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:00 2022] PUT /identity/v3/projects/4b8d975b0a7a41ad81d8842b5d50aa69/users/19b2e84cf05d43c9a644dd8ff983826e/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:00.974235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b556-d8fd-4b57-aca1-59c81ffe4291 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:00.974712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b556-d8fd-4b57-aca1-59c81ffe4291 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:00.975677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b556-d8fd-4b57-aca1-59c81ffe4291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:01.002289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72837798-c5ec-4daf-a90a-02d70c5de4aa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:01.067206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8e05b556-d8fd-4b57-aca1-59c81ffe4291 None None] MFA Rules not processed for user `19b2e84cf05d43c9a644dd8ff983826e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:01.094495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e05b556-d8fd-4b57-aca1-59c81ffe4291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:01.132282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:01.133647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:01.135198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:01.135838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:01.147331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:01.319778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:01.322131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:01.349785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 708/1420] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:01.351894 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:01.355203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 713/1421] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:01.356604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:01.374309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05b556-d8fd-4b57-aca1-59c81ffe4291 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:01.385999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e05b556-d8fd-4b57-aca1-59c81ffe4291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:01.389783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5c108-271a-415c-943e-f3df513d2a83 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:01.407740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:01.408782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:01.409335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:01.409755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] PATH_INFO: `/v3/users/aa808ff135f6439e8c18929da619e494` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:01.412103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa808ff135f6439e8c18929da619e494)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:01.414237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:01.534927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa808ff135f6439e8c18929da619e494'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:01.536427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa808ff135f6439e8c18929da619e494'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:01.562488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa808ff135f6439e8c18929da619e494'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:01.582925 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05b556-d8fd-4b57-aca1-59c81ffe4291 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:01.596730 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ee257-bbcb-47da-9d6a-ad00b08ca08a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:01.597372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee257-bbcb-47da-9d6a-ad00b08ca08a service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:01.600033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee257-bbcb-47da-9d6a-ad00b08ca08a service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:01.600356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee257-bbcb-47da-9d6a-ad00b08ca08a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:01.605712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa808ff135f6439e8c18929da619e494'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:01.606730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b131784-fd9e-4dfc-b7ee-c6c75084d94e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa808ff135f6439e8c18929da619e494'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:01.610462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-472ee257-bbcb-47da-9d6a-ad00b08ca08a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:01.623503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:01.640533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 709/1422] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:01 2022] DELETE /identity/v3/users/aa808ff135f6439e8c18929da619e494 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:01.642858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947b89-9d9e-4f25-93da-5723b7cfb798 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:01.648965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947b89-9d9e-4f25-93da-5723b7cfb798 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:01.649601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947b89-9d9e-4f25-93da-5723b7cfb798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:01.652685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ee257-bbcb-47da-9d6a-ad00b08ca08a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:01.658798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ee257-bbcb-47da-9d6a-ad00b08ca08a service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:01.706512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 714/1423] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:01.710808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a5947b89-9d9e-4f25-93da-5723b7cfb798 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:01.714049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a5947b89-9d9e-4f25-93da-5723b7cfb798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:01.786691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517219dd-b307-4ebd-b213-43bbc4a30cf5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:01.786980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517219dd-b307-4ebd-b213-43bbc4a30cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:01.787259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517219dd-b307-4ebd-b213-43bbc4a30cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:01.860630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 710/1424] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:01.870094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-517219dd-b307-4ebd-b213-43bbc4a30cf5 None None] MFA Rules not processed for user `6b4b2700c20946ea93a1d20d7dea929f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:01.871408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-517219dd-b307-4ebd-b213-43bbc4a30cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:01.893773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947b89-9d9e-4f25-93da-5723b7cfb798 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:01.895701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a5947b89-9d9e-4f25-93da-5723b7cfb798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:01.988505 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947b89-9d9e-4f25-93da-5723b7cfb798 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:01.996794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:01.997054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:01.997293 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:01.997559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:02.006849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:02.151879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 715/1425] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:02.151879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517219dd-b307-4ebd-b213-43bbc4a30cf5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:02.157977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-517219dd-b307-4ebd-b213-43bbc4a30cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:02.285010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:02.286493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:02.313649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 711/1426] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:02.417416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517219dd-b307-4ebd-b213-43bbc4a30cf5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:02.446443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:02.451902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:02.452249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:02.452537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:02.485080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:02.489688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:02.526227 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:02.546176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 716/1427] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:03.377284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:03.379474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:03.411122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fb7a9-35c7-4dd6-93b3-38c8074aebad service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:03.447477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:03.448131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:03.448406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:03.448677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] PATH_INFO: `/v3/projects/bf56dfd9e30c476bba8b1e11b7e0f336` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:03.453161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf56dfd9e30c476bba8b1e11b7e0f336)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:03.465903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:03.534692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:03.539516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:03.606638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf56dfd9e30c476bba8b1e11b7e0f336', 'request_id': 'req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:03.608079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:03.616858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf56dfd9e30c476bba8b1e11b7e0f336', 'request_id': 'req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:03.634794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf56dfd9e30c476bba8b1e11b7e0f336', 'request_id': 'req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:03.673718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6de8e-b520-43eb-b800-eab6809c0549 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:03.685191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] The token cache is being invalidate because project bf56dfd9e30c476bba8b1e11b7e0f336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:03.686441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:03.689149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1e72f1eb-24a6-4e9d-89c8-a7948abe8077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:03.702461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 712/1428] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:03 2022] DELETE /identity/v3/projects/bf56dfd9e30c476bba8b1e11b7e0f336 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:03.710217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51740-744e-4419-9cda-5e7fd8da3bfd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:03.713662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51740-744e-4419-9cda-5e7fd8da3bfd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:03.719161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51740-744e-4419-9cda-5e7fd8da3bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:03.802541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-27d51740-744e-4419-9cda-5e7fd8da3bfd None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:03.807338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-27d51740-744e-4419-9cda-5e7fd8da3bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:03.901889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:03.903451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:03.903834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:03.904094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] PATH_INFO: `/v3/projects/fb2e0425c492454cb57df643f58fb959` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:03.908958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2e0425c492454cb57df643f58fb959)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:03.911515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:04.014373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 713/1429] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:04.014373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8e4b-7737-4a3d-b099-b7d311d9c284 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:04.014373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8e4b-7737-4a3d-b099-b7d311d9c284 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:04.014373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8e4b-7737-4a3d-b099-b7d311d9c284 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:04.014800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 714/1430] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:25:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:25:04.019320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7f4a2-52fa-40e1-a8e4-ad45d3da08f9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:04.020158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7f4a2-52fa-40e1-a8e4-ad45d3da08f9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:04.021013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7f4a2-52fa-40e1-a8e4-ad45d3da08f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:04.061812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2e0425c492454cb57df643f58fb959', 'request_id': 'req-84680f0a-a646-4134-a774-e5b366089d86'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.082504 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2e0425c492454cb57df643f58fb959', 'request_id': 'req-84680f0a-a646-4134-a774-e5b366089d86'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.085495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2eb7f4a2-52fa-40e1-a8e4-ad45d3da08f9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:04.089323 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2eb7f4a2-52fa-40e1-a8e4-ad45d3da08f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:04.099083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2e0425c492454cb57df643f58fb959', 'request_id': 'req-84680f0a-a646-4134-a774-e5b366089d86'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.136238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] The token cache is being invalidate because project fb2e0425c492454cb57df643f58fb959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:04.138842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.140737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-84680f0a-a646-4134-a774-e5b366089d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.144460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 717/1431] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:03 2022] DELETE /identity/v3/projects/fb2e0425c492454cb57df643f58fb959 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:04.150304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cb7fd-d29e-4b8d-a0c6-8bae14b4b8d2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:04.151578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cb7fd-d29e-4b8d-a0c6-8bae14b4b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:04.152527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cb7fd-d29e-4b8d-a0c6-8bae14b4b8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:04.277364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e50cb7fd-d29e-4b8d-a0c6-8bae14b4b8d2 None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:04.281043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e50cb7fd-d29e-4b8d-a0c6-8bae14b4b8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:04.392415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 715/1432] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:04.392415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7f4a2-52fa-40e1-a8e4-ad45d3da08f9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:04.396759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2eb7f4a2-52fa-40e1-a8e4-ad45d3da08f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:04.431544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7f4a2-52fa-40e1-a8e4-ad45d3da08f9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:04.446833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:04.447767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:04.448058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:04.450225 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] PATH_INFO: `/v3/users/35acaf1f6c4548a4a95ff42882ab4bc7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:04.452977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=35acaf1f6c4548a4a95ff42882ab4bc7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:04.455188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:04.517091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 718/1433] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:04.517816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cb7fd-d29e-4b8d-a0c6-8bae14b4b8d2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:04.554359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cb7fd-d29e-4b8d-a0c6-8bae14b4b8d2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:04.561703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:04.562463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:04.562849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:04.565593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:04.566737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:04.567068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:04.575638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:04.607706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 719/1434] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:04.608164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35acaf1f6c4548a4a95ff42882ab4bc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.608164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35acaf1f6c4548a4a95ff42882ab4bc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.608592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:04.612942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:04.615887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35acaf1f6c4548a4a95ff42882ab4bc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.642911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35acaf1f6c4548a4a95ff42882ab4bc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.644274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35acaf1f6c4548a4a95ff42882ab4bc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:04.649334 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:04.677968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 716/1435] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:04 2022] DELETE /identity/v3/users/35acaf1f6c4548a4a95ff42882ab4bc7 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:04.705426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa4b2-f787-4526-8f6f-60d2434e4968 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:04.751798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:05.050098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2690fd-64c2-4734-8faa-7a7007e60e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:05.053617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2690fd-64c2-4734-8faa-7a7007e60e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:05.053984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2690fd-64c2-4734-8faa-7a7007e60e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:05.056847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2690fd-64c2-4734-8faa-7a7007e60e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:05.062486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2690fd-64c2-4734-8faa-7a7007e60e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:05.065677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2690fd-64c2-4734-8faa-7a7007e60e40 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:05.157884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:05.159622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:05.160961 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 720/1436] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:04 2022] POST /identity/v3/projects => generated 387 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:05.165744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2690fd-64c2-4734-8faa-7a7007e60e40 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:05.195177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897cd32-6bd6-4c16-8c14-58b70b8656cb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:05.397612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:05.398838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:05.401923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:05.402268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] PATH_INFO: `/v3/projects/0dc60cbdb9f34257b4b29684cc3584ca` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:05.420785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc60cbdb9f34257b4b29684cc3584ca)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:05.426802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:05.495950 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2690fd-64c2-4734-8faa-7a7007e60e40 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:05.515037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:05.517946 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:05.518730 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:05.520400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:05.521790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:05.525354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:05.626043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 721/1437] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:05 2022] POST /identity/v3/users => generated 294 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:05.634466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:05.639308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:05.639778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc60cbdb9f34257b4b29684cc3584ca', 'request_id': 'req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:05.667828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc60cbdb9f34257b4b29684cc3584ca', 'request_id': 'req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:05.692821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc60cbdb9f34257b4b29684cc3584ca', 'request_id': 'req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:05.728394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] The token cache is being invalidate because project 0dc60cbdb9f34257b4b29684cc3584ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:05.728679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:05.736911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:05.749676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 717/1438] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:05 2022] DELETE /identity/v3/projects/0dc60cbdb9f34257b4b29684cc3584ca => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:05.751555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:05.759500 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:06.018246 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d36c2-77b3-42ec-abc9-88cda6bb2aad admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:06.035686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462819f6-58bc-4120-8f37-fdc9c1c660e3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:06.051533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:06.052136 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:06.052415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:06.052688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:06.053349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:06.055602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:06.102116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:06.139836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 718/1439] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:06.142734 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:06.154435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:06.208251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6355e0-3ffd-426d-905d-e3284f28b99a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:06.229909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:06.230711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:06.231038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:06.231393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] PATH_INFO: `/v3/users/c4299b7ca8fb4be2a3974463ed36a4dc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:06.233265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4299b7ca8fb4be2a3974463ed36a4dc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:06.235257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:06.252183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:06.254377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:06.255552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:06.256144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:06.263971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:06.291409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:06.294051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:06.298172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:06.340745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 722/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:06.341878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:06.362620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b72219-de89-4e86-8cb2-f1216749b157 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:06.369394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95d11b-fe2f-4183-9986-7c7227118877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:06.370636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95d11b-fe2f-4183-9986-7c7227118877 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:06.371108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95d11b-fe2f-4183-9986-7c7227118877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:06.371928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95d11b-fe2f-4183-9986-7c7227118877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:06.372597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95d11b-fe2f-4183-9986-7c7227118877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:06.375092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95d11b-fe2f-4183-9986-7c7227118877 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:06.392631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 723/1441] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:06.397309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95d11b-fe2f-4183-9986-7c7227118877 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:06.403106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4299b7ca8fb4be2a3974463ed36a4dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:06.404133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4299b7ca8fb4be2a3974463ed36a4dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:06.434874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4299b7ca8fb4be2a3974463ed36a4dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:06.439028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95d11b-fe2f-4183-9986-7c7227118877 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:06.470443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:06.470443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:06.470443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:06.470443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] PATH_INFO: `/v3/projects/ff88b874f3d0498bb241e5b4deeb32cf/users/20feb5d2ecc644629a46b7dd19cb29dc/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:06.487858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4299b7ca8fb4be2a3974463ed36a4dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:06.488610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41080e3e-12ca-473e-80c4-9c1018d64842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4299b7ca8fb4be2a3974463ed36a4dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:06.521601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 719/1442] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:06 2022] DELETE /identity/v3/users/c4299b7ca8fb4be2a3974463ed36a4dc => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:06.536108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff88b874f3d0498bb241e5b4deeb32cf, user_id=20feb5d2ecc644629a46b7dd19cb29dc, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:06.540311 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:06.605668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 724/1443] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:06 2022] PUT /identity/v3/projects/ff88b874f3d0498bb241e5b4deeb32cf/users/20feb5d2ecc644629a46b7dd19cb29dc/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:06.608213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6523-fe24-4296-8482-81a33b301f31 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:06.608537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6523-fe24-4296-8482-81a33b301f31 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:06.608809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6523-fe24-4296-8482-81a33b301f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:06.739348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5f5b6523-fe24-4296-8482-81a33b301f31 None None] MFA Rules not processed for user `20feb5d2ecc644629a46b7dd19cb29dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:06.740594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5f5b6523-fe24-4296-8482-81a33b301f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:07.119422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:07.120948 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 720/1444] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 514 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:07.123643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:07.294234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fce42a-905a-48f1-bd2c-d318ee0f749b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:07.300559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97554461-0a39-4129-8c8c-3d625124f74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:07.302215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554461-0a39-4129-8c8c-3d625124f74d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:07.303090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554461-0a39-4129-8c8c-3d625124f74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:07.304918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554461-0a39-4129-8c8c-3d625124f74d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:07.305894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97554461-0a39-4129-8c8c-3d625124f74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:07.309093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97554461-0a39-4129-8c8c-3d625124f74d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:07.456212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 725/1445] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:07 2022] POST /identity/v3/projects => generated 383 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:07.459375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b6523-fe24-4296-8482-81a33b301f31 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:07.480528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b6523-fe24-4296-8482-81a33b301f31 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:07.671115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fb309-b9d8-4d55-8671-aeb05c503d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:07.671750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fb309-b9d8-4d55-8671-aeb05c503d55 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:07.672031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fb309-b9d8-4d55-8671-aeb05c503d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:07.672399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fb309-b9d8-4d55-8671-aeb05c503d55 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:07.673083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fb309-b9d8-4d55-8671-aeb05c503d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:07.681592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fb309-b9d8-4d55-8671-aeb05c503d55 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:07.759027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 721/1446] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:07 2022] POST /identity/v3/users => generated 292 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:07.763511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97554461-0a39-4129-8c8c-3d625124f74d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:07.794797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97554461-0a39-4129-8c8c-3d625124f74d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:07.814447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ba2e7-2f8b-4e6b-913d-7ba039d6affc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:07.816191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ba2e7-2f8b-4e6b-913d-7ba039d6affc admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:07.817342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ba2e7-2f8b-4e6b-913d-7ba039d6affc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:07.818856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ba2e7-2f8b-4e6b-913d-7ba039d6affc admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:07.822944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ba2e7-2f8b-4e6b-913d-7ba039d6affc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:07.826338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ba2e7-2f8b-4e6b-913d-7ba039d6affc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:07.845058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 726/1447] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:07.849065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fb309-b9d8-4d55-8671-aeb05c503d55 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:07.884041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fb309-b9d8-4d55-8671-aeb05c503d55 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:07.921258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eed8ff-63a7-4ff8-bf36-5a7afc8b5f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:07.924982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eed8ff-63a7-4ff8-bf36-5a7afc8b5f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:07.926558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eed8ff-63a7-4ff8-bf36-5a7afc8b5f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:07.930693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eed8ff-63a7-4ff8-bf36-5a7afc8b5f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:07.931900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eed8ff-63a7-4ff8-bf36-5a7afc8b5f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:07.943452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eed8ff-63a7-4ff8-bf36-5a7afc8b5f71 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:07.956989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:07.997341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 722/1448] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:08.001602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ba2e7-2f8b-4e6b-913d-7ba039d6affc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:08.023359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ba2e7-2f8b-4e6b-913d-7ba039d6affc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:08.042963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c42fb8-cbe4-4387-95f8-b94bda09298e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:08.045334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42fb8-cbe4-4387-95f8-b94bda09298e admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:08.047246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42fb8-cbe4-4387-95f8-b94bda09298e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:08.047814 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42fb8-cbe4-4387-95f8-b94bda09298e admin admin] PATH_INFO: `/v3/projects/b4fec7422d47439daebf8cdb9afdb6c5/users/cc40dc2bb2a643ce8e485a5d6a3932cf/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:08.089891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c42fb8-cbe4-4387-95f8-b94bda09298e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4fec7422d47439daebf8cdb9afdb6c5, user_id=cc40dc2bb2a643ce8e485a5d6a3932cf, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:08.092991 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c42fb8-cbe4-4387-95f8-b94bda09298e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:08.130073 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 727/1449] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:08 2022] PUT /identity/v3/projects/b4fec7422d47439daebf8cdb9afdb6c5/users/cc40dc2bb2a643ce8e485a5d6a3932cf/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:08.140868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784406ce-29b0-4db9-ae25-f0b42800698b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:08.141315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784406ce-29b0-4db9-ae25-f0b42800698b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:08.141714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784406ce-29b0-4db9-ae25-f0b42800698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:08.213049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-784406ce-29b0-4db9-ae25-f0b42800698b None None] MFA Rules not processed for user `cc40dc2bb2a643ce8e485a5d6a3932cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:08.216868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-784406ce-29b0-4db9-ae25-f0b42800698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:08.341133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 723/1450] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:08.345362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c42fb8-cbe4-4387-95f8-b94bda09298e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:08.384816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:08.469036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c42fb8-cbe4-4387-95f8-b94bda09298e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:08.492810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5d634-f900-4944-a384-58d7984529b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:08.495413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5d634-f900-4944-a384-58d7984529b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:08.496300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5d634-f900-4944-a384-58d7984529b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:08.496661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5d634-f900-4944-a384-58d7984529b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:08.499947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5d634-f900-4944-a384-58d7984529b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:08.517785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5d634-f900-4944-a384-58d7984529b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:08.678914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784406ce-29b0-4db9-ae25-f0b42800698b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:08.681617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-784406ce-29b0-4db9-ae25-f0b42800698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:08.706740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 728/1451] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:08 2022] POST /identity/v3/projects => generated 381 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:08.709536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5d634-f900-4944-a384-58d7984529b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:08.774881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784406ce-29b0-4db9-ae25-f0b42800698b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:09.020974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5d634-f900-4944-a384-58d7984529b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:09.033983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:09.035010 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.035542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.036049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.036913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:09.046203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:09.085556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:09.095330 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.095418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.095645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] PATH_INFO: `/v3/projects/d921d8edfabf4a39934705b40d20c0c8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.110272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d921d8edfabf4a39934705b40d20c0c8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:09.138567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:09.165870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 729/1452] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:08 2022] POST /identity/v3/users => generated 297 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:09.167518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:09.177448 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:09.221175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c7fd3-9d8f-4c5b-b0f8-43ff0c7c0ad2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:09.245838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:09.247527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.248046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.248527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] PATH_INFO: `/v3/users/bfce506069304a52932369a4d3770c95` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.252026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfce506069304a52932369a4d3770c95)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:09.257471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:09.322398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd921d8edfabf4a39934705b40d20c0c8', 'request_id': 'req-21d635fc-e16d-4586-ae03-263c6f4d6848'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.342900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd921d8edfabf4a39934705b40d20c0c8', 'request_id': 'req-21d635fc-e16d-4586-ae03-263c6f4d6848'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.357012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd921d8edfabf4a39934705b40d20c0c8', 'request_id': 'req-21d635fc-e16d-4586-ae03-263c6f4d6848'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.380427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfce506069304a52932369a4d3770c95'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.383349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] The token cache is being invalidate because project d921d8edfabf4a39934705b40d20c0c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:09.383653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.385447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.393222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 724/1453] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:08 2022] DELETE /identity/v3/projects/d921d8edfabf4a39934705b40d20c0c8 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:09.394374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:09.397150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:09.399471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfce506069304a52932369a4d3770c95'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.444300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfce506069304a52932369a4d3770c95'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.460150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:09.462709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfce506069304a52932369a4d3770c95'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.463555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-623c3c5c-241a-4c5d-b81e-973de2a92e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfce506069304a52932369a4d3770c95'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:09.502984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 730/1454] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:09 2022] DELETE /identity/v3/users/bfce506069304a52932369a4d3770c95 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:09.506508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b543f6c-03a7-4c8f-940e-d255913f4652 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.506896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b543f6c-03a7-4c8f-940e-d255913f4652 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.507199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b543f6c-03a7-4c8f-940e-d255913f4652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.564200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3b543f6c-03a7-4c8f-940e-d255913f4652 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:09.565772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3b543f6c-03a7-4c8f-940e-d255913f4652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:09.590714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:09.597186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d635fc-e16d-4586-ae03-263c6f4d6848 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:09.776005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 731/1455] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:09.803520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec5b6-f1bb-45d2-a2d9-6470a78628b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:09.804145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec5b6-f1bb-45d2-a2d9-6470a78628b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.804960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec5b6-f1bb-45d2-a2d9-6470a78628b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.805236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec5b6-f1bb-45d2-a2d9-6470a78628b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.806039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ec5b6-f1bb-45d2-a2d9-6470a78628b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:09.807720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05ec8f-59e4-48fa-bdce-52c0a07672b2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.807954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ec5b6-f1bb-45d2-a2d9-6470a78628b2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:09.811087 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05ec8f-59e4-48fa-bdce-52c0a07672b2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.813503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05ec8f-59e4-48fa-bdce-52c0a07672b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.828096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 725/1456] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:09.830299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4be928-09f9-4f2c-a895-035d7175fb80 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.830931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4be928-09f9-4f2c-a895-035d7175fb80 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.831510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4be928-09f9-4f2c-a895-035d7175fb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.848005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ce05ec8f-59e4-48fa-bdce-52c0a07672b2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:09.849239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ce05ec8f-59e4-48fa-bdce-52c0a07672b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:09.874346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 732/1457] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:09.875275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05ec8f-59e4-48fa-bdce-52c0a07672b2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:09.879029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3a4be928-09f9-4f2c-a895-035d7175fb80 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:09.880538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3a4be928-09f9-4f2c-a895-035d7175fb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:09.895423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 726/1458] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:09.899857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05ec8f-59e4-48fa-bdce-52c0a07672b2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:09.907640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4be928-09f9-4f2c-a895-035d7175fb80 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:09.912201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:09.912918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.913212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.913509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] PATH_INFO: `/v3/projects/5e1b0bd5323e419a824dfd333445d775/users/682a30be74ac430d95b8ef1f9a44bd2a/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.922502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4be928-09f9-4f2c-a895-035d7175fb80 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:09.930692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:09.931644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:09.932112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:09.932612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:09.933702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:09.934246 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:09.936140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:09.966409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 727/1459] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:09.968350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:09.970893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:10.061510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c9da-3f73-4119-9050-b5374fb7397a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:10.068575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cf2b1-b948-4272-8615-a53c741300c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:10.069907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cf2b1-b948-4272-8615-a53c741300c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:10.071941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cf2b1-b948-4272-8615-a53c741300c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:10.072545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cf2b1-b948-4272-8615-a53c741300c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:10.088553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67cf2b1-b948-4272-8615-a53c741300c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:10.090481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67cf2b1-b948-4272-8615-a53c741300c2 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:10.114709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 728/1460] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:10.116678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cf2b1-b948-4272-8615-a53c741300c2 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:10.161292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cf2b1-b948-4272-8615-a53c741300c2 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:10.199560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6745f-a754-4c8c-a480-7fa7b137f767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:10.201282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6745f-a754-4c8c-a480-7fa7b137f767 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:10.201893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6745f-a754-4c8c-a480-7fa7b137f767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:10.202958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6745f-a754-4c8c-a480-7fa7b137f767 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:10.203296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1b0bd5323e419a824dfd333445d775, user_id=682a30be74ac430d95b8ef1f9a44bd2a, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:10.204270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6745f-a754-4c8c-a480-7fa7b137f767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:10.207190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:10.208751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6745f-a754-4c8c-a480-7fa7b137f767 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:10.260604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 733/1461] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:09 2022] PUT /identity/v3/projects/5e1b0bd5323e419a824dfd333445d775/users/682a30be74ac430d95b8ef1f9a44bd2a/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:10.263141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:10.272072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:10.286412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 729/1462] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:10 2022] POST /identity/v3/projects => generated 393 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:10.286412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6745f-a754-4c8c-a480-7fa7b137f767 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:10.367108 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:10.410409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7115d-3fba-422e-ade6-aa508e32983a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:10.462855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6745f-a754-4c8c-a480-7fa7b137f767 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:10.470720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9917e1-35cc-4186-a46b-5cbbd3b6627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:10.471583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9917e1-35cc-4186-a46b-5cbbd3b6627f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:10.474571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9917e1-35cc-4186-a46b-5cbbd3b6627f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:10.475066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9917e1-35cc-4186-a46b-5cbbd3b6627f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:10.476127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9917e1-35cc-4186-a46b-5cbbd3b6627f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:10.479862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9917e1-35cc-4186-a46b-5cbbd3b6627f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:10.515630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 730/1463] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:10.515630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9917e1-35cc-4186-a46b-5cbbd3b6627f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:10.542171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9917e1-35cc-4186-a46b-5cbbd3b6627f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:10.556570 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ea60c-07a1-433b-9ebe-35e373dfa1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:10.557164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea60c-07a1-433b-9ebe-35e373dfa1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:10.558029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea60c-07a1-433b-9ebe-35e373dfa1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:10.558309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea60c-07a1-433b-9ebe-35e373dfa1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:10.560205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ea60c-07a1-433b-9ebe-35e373dfa1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:10.560960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:10.561848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:10.562189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ea60c-07a1-433b-9ebe-35e373dfa1b6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:10.562583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:10.563048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] PATH_INFO: `/v3/projects/45e9984c466947e3a537cde94dd36c62` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:10.564545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e9984c466947e3a537cde94dd36c62)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:10.567047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:10.624097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:10.632167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 731/1464] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:10 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:10.633058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ea60c-07a1-433b-9ebe-35e373dfa1b6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:10.661010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ea60c-07a1-433b-9ebe-35e373dfa1b6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:10.676359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e9984c466947e3a537cde94dd36c62', 'request_id': 'req-c1de02c1-c3be-4189-a1a4-38ba924365d5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:10.686531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e9984c466947e3a537cde94dd36c62', 'request_id': 'req-c1de02c1-c3be-4189-a1a4-38ba924365d5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:10.709792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e9984c466947e3a537cde94dd36c62', 'request_id': 'req-c1de02c1-c3be-4189-a1a4-38ba924365d5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:10.755908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] The token cache is being invalidate because project 45e9984c466947e3a537cde94dd36c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:10.756098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:10.757341 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:10.759799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 734/1465] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:10 2022] DELETE /identity/v3/projects/45e9984c466947e3a537cde94dd36c62 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:10.761029 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:10.762604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:10.770271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383017e-fc5d-4276-b97d-90731691b068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:10.771541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383017e-fc5d-4276-b97d-90731691b068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:10.771834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383017e-fc5d-4276-b97d-90731691b068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:10.772111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383017e-fc5d-4276-b97d-90731691b068 admin admin] PATH_INFO: `/v3/projects/5e1b0bd5323e419a824dfd333445d775/users/682a30be74ac430d95b8ef1f9a44bd2a/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:10.774977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383017e-fc5d-4276-b97d-90731691b068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1b0bd5323e419a824dfd333445d775, user_id=682a30be74ac430d95b8ef1f9a44bd2a, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:10.779474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383017e-fc5d-4276-b97d-90731691b068 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:10.796846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 732/1466] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:10 2022] PUT /identity/v3/projects/5e1b0bd5323e419a824dfd333445d775/users/682a30be74ac430d95b8ef1f9a44bd2a/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:10.798312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fa24-d7a9-46e7-aacd-a79f276fe535 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:10.798699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fa24-d7a9-46e7-aacd-a79f276fe535 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:10.799203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fa24-d7a9-46e7-aacd-a79f276fe535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:10.846346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1269fa24-d7a9-46e7-aacd-a79f276fe535 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:10.849508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1269fa24-d7a9-46e7-aacd-a79f276fe535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:10.911496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de02c1-c3be-4189-a1a4-38ba924365d5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:11.020403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 733/1467] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:11.022062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a0c1f-d991-437e-b6a6-972d31784c93 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.022599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a0c1f-d991-437e-b6a6-972d31784c93 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.023099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a0c1f-d991-437e-b6a6-972d31784c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.081438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4abb82-2685-4014-93a0-c8287d472e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:11.082080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4abb82-2685-4014-93a0-c8287d472e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.082378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4abb82-2685-4014-93a0-c8287d472e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.082654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4abb82-2685-4014-93a0-c8287d472e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.083375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4abb82-2685-4014-93a0-c8287d472e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:11.085191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4abb82-2685-4014-93a0-c8287d472e0a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:11.103111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-095a0c1f-d991-437e-b6a6-972d31784c93 None None] MFA Rules not processed for user `682a30be74ac430d95b8ef1f9a44bd2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:11.104428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-095a0c1f-d991-437e-b6a6-972d31784c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:11.109786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 735/1468] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:11.111203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c59da-002e-44aa-861d-c6e3f7985b40 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.111521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c59da-002e-44aa-861d-c6e3f7985b40 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.111815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c59da-002e-44aa-861d-c6e3f7985b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.150668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:11.164250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ce1c59da-002e-44aa-861d-c6e3f7985b40 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:11.166545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ce1c59da-002e-44aa-861d-c6e3f7985b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:11.191698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 736/1469] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:11.192710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d04f88-7168-464e-a881-54dd24519baa None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.193964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d04f88-7168-464e-a881-54dd24519baa None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.195656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d04f88-7168-464e-a881-54dd24519baa None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.198725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 737/1470] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:25:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:25:11.199576 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d04f88-7168-464e-a881-54dd24519baa None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:11.227276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d04f88-7168-464e-a881-54dd24519baa None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:11.238832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1e376-f64c-400b-aaec-b4cb5cf601ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:11.241011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1e376-f64c-400b-aaec-b4cb5cf601ba admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.242852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1e376-f64c-400b-aaec-b4cb5cf601ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.243140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1e376-f64c-400b-aaec-b4cb5cf601ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.243826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1e376-f64c-400b-aaec-b4cb5cf601ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:11.245474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1e376-f64c-400b-aaec-b4cb5cf601ba admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:11.252250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 734/1471] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:11.257892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a0c1f-d991-437e-b6a6-972d31784c93 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:11.276134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 738/1472] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:11.277773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a064ac-f2b3-4d69-a6e9-a960bf5860a5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.278057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a064ac-f2b3-4d69-a6e9-a960bf5860a5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.278342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a064ac-f2b3-4d69-a6e9-a960bf5860a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.284136 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a0c1f-d991-437e-b6a6-972d31784c93 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:11.292961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:11.293596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.293880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.294151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.294849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:11.295152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:11.297917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:11.322159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 735/1473] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:11.322389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-99a064ac-f2b3-4d69-a6e9-a960bf5860a5 None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:11.323128 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:11.323834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-99a064ac-f2b3-4d69-a6e9-a960bf5860a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:11.325069 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:11.414774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:11.487734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 739/1474] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:25:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:11.488425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a064ac-f2b3-4d69-a6e9-a960bf5860a5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:11.511893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcc638-1628-4c9c-8974-3cf005ab7120 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:11.512086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a064ac-f2b3-4d69-a6e9-a960bf5860a5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:11.521138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7b852-ee81-4f9f-846f-6c109c779fcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:11.521784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7b852-ee81-4f9f-846f-6c109c779fcb service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.522039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7b852-ee81-4f9f-846f-6c109c779fcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.522389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7b852-ee81-4f9f-846f-6c109c779fcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.525045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:11.526084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.526194 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.526544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] PATH_INFO: `/v3/projects/613caad261024870aac96b9d28a90bac/users/4fe615abb63e425f88ac26265311eaa9/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.551569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7b852-ee81-4f9f-846f-6c109c779fcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:11.553338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7b852-ee81-4f9f-846f-6c109c779fcb service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:11.567021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 736/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:11.567877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7b852-ee81-4f9f-846f-6c109c779fcb service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:11.569494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] RBAC: Authorizing `identity:create_grant(project_id=613caad261024870aac96b9d28a90bac, user_id=4fe615abb63e425f88ac26265311eaa9, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:11.572907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:11.595656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7b852-ee81-4f9f-846f-6c109c779fcb service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:11.612479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 740/1476] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:11 2022] PUT /identity/v3/projects/613caad261024870aac96b9d28a90bac/users/4fe615abb63e425f88ac26265311eaa9/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:11.613013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b480dd-4dca-4b9d-98a0-b0a17b60aa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:11.613360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:11.613711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b480dd-4dca-4b9d-98a0-b0a17b60aa8c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.613917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b480dd-4dca-4b9d-98a0-b0a17b60aa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.614217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b480dd-4dca-4b9d-98a0-b0a17b60aa8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.614887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b480dd-4dca-4b9d-98a0-b0a17b60aa8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:11.616788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b480dd-4dca-4b9d-98a0-b0a17b60aa8c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:11.626442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:11.681038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 737/1477] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:11 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:11.683141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23333-3140-434c-afa7-40b1389a88f6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:11.683492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23333-3140-434c-afa7-40b1389a88f6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:11.683736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23333-3140-434c-afa7-40b1389a88f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:11.784678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0dc23333-3140-434c-afa7-40b1389a88f6 None None] MFA Rules not processed for user `4fe615abb63e425f88ac26265311eaa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:11.785935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0dc23333-3140-434c-afa7-40b1389a88f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:11.807310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad367726-7731-48f4-bcf6-6b131bd7efde admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.007082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 738/1478] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:12.008140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc23333-3140-434c-afa7-40b1389a88f6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.066697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:12.093516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058e86a-9ca3-453e-836d-5f359c7d248a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.094178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058e86a-9ca3-453e-836d-5f359c7d248a service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.094762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058e86a-9ca3-453e-836d-5f359c7d248a service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.095066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058e86a-9ca3-453e-836d-5f359c7d248a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.106280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058e86a-9ca3-453e-836d-5f359c7d248a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.108053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058e86a-9ca3-453e-836d-5f359c7d248a service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:12.140587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 741/1479] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:12.143049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058e86a-9ca3-453e-836d-5f359c7d248a service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.226484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc23333-3140-434c-afa7-40b1389a88f6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.246041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.247553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.248145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.248714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.249602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.251782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:12.258775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:12.300282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 739/1480] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:12 2022] POST /identity/v3/users => generated 285 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:12.301488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.305677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:12.330185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b0c51-84f0-4345-ab1a-8fd014e6b587 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.341200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.342472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058e86a-9ca3-453e-836d-5f359c7d248a service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.342839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.343360 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.343822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] PATH_INFO: `/v3/projects/750b570dc8cd4ca484d8d88727495826` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.345388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750b570dc8cd4ca484d8d88727495826)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.347717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:12.358858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8482f-7dc7-4162-8ff3-b8f235c471a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.359580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8482f-7dc7-4162-8ff3-b8f235c471a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.359866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8482f-7dc7-4162-8ff3-b8f235c471a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.360200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8482f-7dc7-4162-8ff3-b8f235c471a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.360863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8482f-7dc7-4162-8ff3-b8f235c471a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.364422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8482f-7dc7-4162-8ff3-b8f235c471a4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:12.420593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 742/1481] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:12 2022] POST /identity/v3/projects => generated 393 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:12.420593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8482f-7dc7-4162-8ff3-b8f235c471a4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.479472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750b570dc8cd4ca484d8d88727495826', 'request_id': 'req-d426295f-5dd9-4df7-b2e7-03cca88299b1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.496499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750b570dc8cd4ca484d8d88727495826', 'request_id': 'req-d426295f-5dd9-4df7-b2e7-03cca88299b1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.514925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750b570dc8cd4ca484d8d88727495826', 'request_id': 'req-d426295f-5dd9-4df7-b2e7-03cca88299b1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.537761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8482f-7dc7-4162-8ff3-b8f235c471a4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.547376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] The token cache is being invalidate because project 750b570dc8cd4ca484d8d88727495826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:12.547678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.553559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.558254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 740/1482] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:12 2022] DELETE /identity/v3/projects/750b570dc8cd4ca484d8d88727495826 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:12.561147 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.565526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:12.646477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.650676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.650999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.651301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.652084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.655094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:12.687118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 743/1483] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:12.692445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.697258 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:12.709727 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426295f-5dd9-4df7-b2e7-03cca88299b1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.724352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.724976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.725256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.725526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.726162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.726997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76c47a-56e9-4a66-bc88-e5d962c36bb0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.728769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:12.754404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.760504 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.761027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.761467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] PATH_INFO: `/v3/users/19b2e84cf05d43c9a644dd8ff983826e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.764960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b2e84cf05d43c9a644dd8ff983826e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.770114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 741/1484] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:12 2022] POST /identity/v3/users => generated 303 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:12.770627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:12.771077 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.772705 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:12.787038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3489bbf-8e4c-453f-a038-80e77525947c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.794125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.795019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.795377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.795651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.796496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.798471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:12.814465 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 742/1485] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:12.815618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.819117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:12.837698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:12.847912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b2e84cf05d43c9a644dd8ff983826e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.849033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b2e84cf05d43c9a644dd8ff983826e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.857300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:12.878533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b2e84cf05d43c9a644dd8ff983826e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.906472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b2e84cf05d43c9a644dd8ff983826e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.907280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b2e84cf05d43c9a644dd8ff983826e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:12.913086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2fb23-d8a0-4003-8e59-a8767530951e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:12.918588 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:12.919292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:12.919616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:12.919888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:12.921800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:12.923466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 744/1486] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:12 2022] DELETE /identity/v3/users/19b2e84cf05d43c9a644dd8ff983826e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:12.924205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:12.925564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:12.991348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:12.993529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.005170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df90cb-96b1-4318-bcbd-a6a8f0374a27 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.013237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfec5-27ca-4839-988b-e8ac8391b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.014161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfec5-27ca-4839-988b-e8ac8391b916 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.014439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 743/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:13.015291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfec5-27ca-4839-988b-e8ac8391b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.015493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.016060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfec5-27ca-4839-988b-e8ac8391b916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.017068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfec5-27ca-4839-988b-e8ac8391b916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.019368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfec5-27ca-4839-988b-e8ac8391b916 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.020238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.032902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 745/1488] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:13.037277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfec5-27ca-4839-988b-e8ac8391b916 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.042932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed8f4f-9008-4d2c-95da-594145bfaee4 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.048066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfec5-27ca-4839-988b-e8ac8391b916 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.048813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9be8-034a-45ce-9848-a1902cb74082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.049634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9be8-034a-45ce-9848-a1902cb74082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.050216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9be8-034a-45ce-9848-a1902cb74082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.050662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9be8-034a-45ce-9848-a1902cb74082 admin admin] PATH_INFO: `/v3/projects/879c14664c7a42e4a5e133b53990462c/users/01c8c33c0d54424e99c23178c862ba64/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.053890 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a955819-577f-4154-9e98-f87fe1fd8424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.054657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a955819-577f-4154-9e98-f87fe1fd8424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.054936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a955819-577f-4154-9e98-f87fe1fd8424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.055205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a955819-577f-4154-9e98-f87fe1fd8424 admin admin] PATH_INFO: `/v3/projects/53d5bf9f77304c168e84b47978aac30c/users/c8a107be45834413a4a31c6dbd230307/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.076106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a9be8-034a-45ce-9848-a1902cb74082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879c14664c7a42e4a5e133b53990462c, user_id=01c8c33c0d54424e99c23178c862ba64, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.079485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a9be8-034a-45ce-9848-a1902cb74082 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.092979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a955819-577f-4154-9e98-f87fe1fd8424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d5bf9f77304c168e84b47978aac30c, user_id=c8a107be45834413a4a31c6dbd230307, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.097259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a955819-577f-4154-9e98-f87fe1fd8424 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.101087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 744/1489] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:13 2022] PUT /identity/v3/projects/879c14664c7a42e4a5e133b53990462c/users/01c8c33c0d54424e99c23178c862ba64/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:13.124201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2ffa-9908-4b22-a1a5-ed667708e22a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.124485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2ffa-9908-4b22-a1a5-ed667708e22a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.124753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2ffa-9908-4b22-a1a5-ed667708e22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.125442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 746/1490] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:13 2022] PUT /identity/v3/projects/53d5bf9f77304c168e84b47978aac30c/users/c8a107be45834413a4a31c6dbd230307/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:13.129382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a955819-577f-4154-9e98-f87fe1fd8424 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.165001 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-aa2f2ffa-9908-4b22-a1a5-ed667708e22a None None] MFA Rules not processed for user `01c8c33c0d54424e99c23178c862ba64`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:13.166829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aa2f2ffa-9908-4b22-a1a5-ed667708e22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.222840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a955819-577f-4154-9e98-f87fe1fd8424 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.230589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60d2b-7717-4e78-b05a-d051a76bf82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.231377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60d2b-7717-4e78-b05a-d051a76bf82a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.231659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60d2b-7717-4e78-b05a-d051a76bf82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.232431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60d2b-7717-4e78-b05a-d051a76bf82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.233481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f60d2b-7717-4e78-b05a-d051a76bf82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.235538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f60d2b-7717-4e78-b05a-d051a76bf82a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.248414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 745/1491] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:13.251494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 747/1492] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:13.252236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2ffa-9908-4b22-a1a5-ed667708e22a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.255327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60d2b-7717-4e78-b05a-d051a76bf82a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.271539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60d2b-7717-4e78-b05a-d051a76bf82a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.275972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2ffa-9908-4b22-a1a5-ed667708e22a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.278268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.279135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.279416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.279687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] PATH_INFO: `/v3/projects/53d5bf9f77304c168e84b47978aac30c/users/c8a107be45834413a4a31c6dbd230307/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.282019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b08e2-8682-4015-9ba2-7ac147ee4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.282538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d5bf9f77304c168e84b47978aac30c, user_id=c8a107be45834413a4a31c6dbd230307, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.282834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b08e2-8682-4015-9ba2-7ac147ee4bce admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.283115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b08e2-8682-4015-9ba2-7ac147ee4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.283393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b08e2-8682-4015-9ba2-7ac147ee4bce admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.284378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b08e2-8682-4015-9ba2-7ac147ee4bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.285255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.287821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b08e2-8682-4015-9ba2-7ac147ee4bce admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.317895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 748/1493] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:13 2022] PUT /identity/v3/projects/53d5bf9f77304c168e84b47978aac30c/users/c8a107be45834413a4a31c6dbd230307/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:13.327229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.330234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 746/1494] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:13 2022] POST /identity/v3/projects => generated 369 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:13.331787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767efe0-ee99-4053-9ac3-d98220d4062a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.332141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767efe0-ee99-4053-9ac3-d98220d4062a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.332658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767efe0-ee99-4053-9ac3-d98220d4062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.342377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.354118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:13.373524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9767efe0-ee99-4053-9ac3-d98220d4062a None None] MFA Rules not processed for user `c8a107be45834413a4a31c6dbd230307`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:13.375673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9767efe0-ee99-4053-9ac3-d98220d4062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.438699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:13.498256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624185bb-02c5-415e-87b5-2c93707078ac admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.551137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 747/1495] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:13.552307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767efe0-ee99-4053-9ac3-d98220d4062a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.586499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.587338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.587621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.587900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] PATH_INFO: `/v3/projects/4b8d975b0a7a41ad81d8842b5d50aa69` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.589195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b8d975b0a7a41ad81d8842b5d50aa69)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.591163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.628741 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767efe0-ee99-4053-9ac3-d98220d4062a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.643306 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8d975b0a7a41ad81d8842b5d50aa69', 'request_id': 'req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:13.649781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8d975b0a7a41ad81d8842b5d50aa69', 'request_id': 'req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:13.657839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8d975b0a7a41ad81d8842b5d50aa69', 'request_id': 'req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:13.668503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] The token cache is being invalidate because project 4b8d975b0a7a41ad81d8842b5d50aa69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:13.668784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:13.669420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:13.670975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 749/1496] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:13 2022] DELETE /identity/v3/projects/4b8d975b0a7a41ad81d8842b5d50aa69 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:13.671740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.673047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.691056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8717c-36fa-4959-a3ab-a1dce70e2c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.691692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8717c-36fa-4959-a3ab-a1dce70e2c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.692054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8717c-36fa-4959-a3ab-a1dce70e2c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.692348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8717c-36fa-4959-a3ab-a1dce70e2c31 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.693007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8717c-36fa-4959-a3ab-a1dce70e2c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.695387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8717c-36fa-4959-a3ab-a1dce70e2c31 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.718867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 748/1497] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:13 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:13.720339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ff6f1-528b-48ca-9284-359cf5984dc4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.720689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ff6f1-528b-48ca-9284-359cf5984dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.720970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ff6f1-528b-48ca-9284-359cf5984dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.741316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb63ebb-d0ef-4356-88db-7b2870fd4b09 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.742622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4d0ff6f1-528b-48ca-9284-359cf5984dc4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:13.743859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4d0ff6f1-528b-48ca-9284-359cf5984dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.745645 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.746251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.746555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.746867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.748857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.752294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 749/1498] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:13.753093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ff6f1-528b-48ca-9284-359cf5984dc4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.754512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4d0ff6f1-528b-48ca-9284-359cf5984dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.767813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ff6f1-528b-48ca-9284-359cf5984dc4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.773275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eb762-44a8-42f9-92c0-eb9630cb66cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.773732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb762-44a8-42f9-92c0-eb9630cb66cf admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.773924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb762-44a8-42f9-92c0-eb9630cb66cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.774190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb762-44a8-42f9-92c0-eb9630cb66cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.774886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2eb762-44a8-42f9-92c0-eb9630cb66cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.776419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2eb762-44a8-42f9-92c0-eb9630cb66cf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.779893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:13.792249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 750/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:13.792249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ebc93-d9fd-4f64-a4ea-ce83a36c8174 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.792249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ebc93-d9fd-4f64-a4ea-ce83a36c8174 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.792249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ebc93-d9fd-4f64-a4ea-ce83a36c8174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.814722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-169ebc93-d9fd-4f64-a4ea-ce83a36c8174 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:13.816109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-169ebc93-d9fd-4f64-a4ea-ce83a36c8174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:13.827984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 751/1500] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:13.827984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ebc93-d9fd-4f64-a4ea-ce83a36c8174 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.833042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:13.835314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.836997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ebc93-d9fd-4f64-a4ea-ce83a36c8174 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.837536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.843605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda9135-524b-4be3-a5de-cf81c2ae2af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.844389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda9135-524b-4be3-a5de-cf81c2ae2af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.844669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda9135-524b-4be3-a5de-cf81c2ae2af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.844945 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda9135-524b-4be3-a5de-cf81c2ae2af8 admin admin] PATH_INFO: `/v3/projects/13a60d74e5864745b34d452ad38c1503/users/c1c10f68800a4adba55338ffbf8ac71f/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.847961 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 750/1501] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:13 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:13.848625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.860502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153ce0a-4b8c-4a34-83e2-ad758e3dfd3d service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:13.866671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:13.867353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:13.867627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:13.867915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:13.868172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda9135-524b-4be3-a5de-cf81c2ae2af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a60d74e5864745b34d452ad38c1503, user_id=c1c10f68800a4adba55338ffbf8ac71f, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.868880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:13.870438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:13.872078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda9135-524b-4be3-a5de-cf81c2ae2af8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.872275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:13.899398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 752/1502] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:13 2022] PUT /identity/v3/projects/13a60d74e5864745b34d452ad38c1503/users/c1c10f68800a4adba55338ffbf8ac71f/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:13.902372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 751/1503] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:13.910349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda9135-524b-4be3-a5de-cf81c2ae2af8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.912910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:13.927252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bec93-2a3d-4ec7-8564-5d08f1574eb2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.008116 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda9135-524b-4be3-a5de-cf81c2ae2af8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.014983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.016370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.016705 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.016976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.017658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.019377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.021001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4e220-a4a7-4ea3-ab6f-da20dd772a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.022288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4e220-a4a7-4ea3-ab6f-da20dd772a25 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.022288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4e220-a4a7-4ea3-ab6f-da20dd772a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.022513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4e220-a4a7-4ea3-ab6f-da20dd772a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.022994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4e220-a4a7-4ea3-ab6f-da20dd772a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.026461 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4e220-a4a7-4ea3-ab6f-da20dd772a25 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.032733 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 753/1504] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:14.033582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:14.035538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:14.087399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 752/1505] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:13 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:14.088413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4e220-a4a7-4ea3-ab6f-da20dd772a25 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:14.156968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2617801-9a30-4e83-b22c-c76aaa3c3db2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.200074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4e220-a4a7-4ea3-ab6f-da20dd772a25 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.214349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d67c1-309f-46e4-87c3-45df98ce0245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.214349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d67c1-309f-46e4-87c3-45df98ce0245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.214349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d67c1-309f-46e4-87c3-45df98ce0245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.214349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d67c1-309f-46e4-87c3-45df98ce0245 admin admin] PATH_INFO: `/v3/projects/13a60d74e5864745b34d452ad38c1503/users/c1c10f68800a4adba55338ffbf8ac71f/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.215471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d67c1-309f-46e4-87c3-45df98ce0245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a60d74e5864745b34d452ad38c1503, user_id=c1c10f68800a4adba55338ffbf8ac71f, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.221601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d67c1-309f-46e4-87c3-45df98ce0245 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.237868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110a3ec-edc6-474f-98dd-7a52a831d814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.239252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110a3ec-edc6-474f-98dd-7a52a831d814 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.239641 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110a3ec-edc6-474f-98dd-7a52a831d814 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.240037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110a3ec-edc6-474f-98dd-7a52a831d814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.250924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 753/1506] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:14 2022] PUT /identity/v3/projects/13a60d74e5864745b34d452ad38c1503/users/c1c10f68800a4adba55338ffbf8ac71f/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:14.250924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d67c1-309f-46e4-87c3-45df98ce0245 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:14.252366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8110a3ec-edc6-474f-98dd-7a52a831d814 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:14.279206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:14.309058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:14.326724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110a3ec-edc6-474f-98dd-7a52a831d814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.328261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110a3ec-edc6-474f-98dd-7a52a831d814 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.329260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d67c1-309f-46e4-87c3-45df98ce0245 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.334062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9856ad3-290e-43aa-9aff-8ddd9d0287a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.334761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856ad3-290e-43aa-9aff-8ddd9d0287a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.335038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856ad3-290e-43aa-9aff-8ddd9d0287a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.335302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856ad3-290e-43aa-9aff-8ddd9d0287a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.336548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9856ad3-290e-43aa-9aff-8ddd9d0287a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.340140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 754/1507] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:14.341568 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084cc31-4720-4e90-9356-9988f9ad7e13 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.341954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084cc31-4720-4e90-9356-9988f9ad7e13 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.342292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084cc31-4720-4e90-9356-9988f9ad7e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.342747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9856ad3-290e-43aa-9aff-8ddd9d0287a8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.396270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 754/1508] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:14 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:14.396952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6084cc31-4720-4e90-9356-9988f9ad7e13 None None] MFA Rules not processed for user `c1c10f68800a4adba55338ffbf8ac71f`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:14.397315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9856ad3-290e-43aa-9aff-8ddd9d0287a8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:14.400455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6084cc31-4720-4e90-9356-9988f9ad7e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:14.410998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9856ad3-290e-43aa-9aff-8ddd9d0287a8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.420406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf73f-a309-4122-801b-124d1be2d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.421431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf73f-a309-4122-801b-124d1be2d083 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.424392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf73f-a309-4122-801b-124d1be2d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.424816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf73f-a309-4122-801b-124d1be2d083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.426023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bf73f-a309-4122-801b-124d1be2d083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.428627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bf73f-a309-4122-801b-124d1be2d083 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.441169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 755/1509] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:14.444881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf73f-a309-4122-801b-124d1be2d083 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:14.456906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf73f-a309-4122-801b-124d1be2d083 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.465810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61a574-402b-4914-be86-11caa89fce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.467241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a574-402b-4914-be86-11caa89fce80 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.467971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a574-402b-4914-be86-11caa89fce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.468650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a574-402b-4914-be86-11caa89fce80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.469637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61a574-402b-4914-be86-11caa89fce80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.471834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61a574-402b-4914-be86-11caa89fce80 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.488482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 756/1510] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:14.492559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61a574-402b-4914-be86-11caa89fce80 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:14.508234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61a574-402b-4914-be86-11caa89fce80 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.516767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c8aaa-ae80-4313-9838-9f168bec028a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.518283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c8aaa-ae80-4313-9838-9f168bec028a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.518850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c8aaa-ae80-4313-9838-9f168bec028a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.519149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c8aaa-ae80-4313-9838-9f168bec028a admin admin] PATH_INFO: `/v3/projects/5f827170fc784bd493d95361d6401d7f/users/3062acca43534df6a3d17af1377e18b1/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.524183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 755/1511] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:14.525441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084cc31-4720-4e90-9356-9988f9ad7e13 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:14.546312 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c8aaa-ae80-4313-9838-9f168bec028a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f827170fc784bd493d95361d6401d7f, user_id=3062acca43534df6a3d17af1377e18b1, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.549861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c8aaa-ae80-4313-9838-9f168bec028a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.570827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 757/1512] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:14 2022] PUT /identity/v3/projects/5f827170fc784bd493d95361d6401d7f/users/3062acca43534df6a3d17af1377e18b1/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:14.581921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7627ef-1316-452a-a107-19db4c9d0635 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.582252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7627ef-1316-452a-a107-19db4c9d0635 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.582571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7627ef-1316-452a-a107-19db4c9d0635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.606867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084cc31-4720-4e90-9356-9988f9ad7e13 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.649138 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0d7627ef-1316-452a-a107-19db4c9d0635 None None] MFA Rules not processed for user `3062acca43534df6a3d17af1377e18b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:14.651293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0d7627ef-1316-452a-a107-19db4c9d0635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:14.710017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56ce28-d77e-412f-8b6d-6ebc8bfddde6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.712914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56ce28-d77e-412f-8b6d-6ebc8bfddde6 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.713718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56ce28-d77e-412f-8b6d-6ebc8bfddde6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.714090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56ce28-d77e-412f-8b6d-6ebc8bfddde6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.774077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 758/1513] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:14.779805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7627ef-1316-452a-a107-19db4c9d0635 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:14.796089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:14.838861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56ce28-d77e-412f-8b6d-6ebc8bfddde6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.840696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56ce28-d77e-412f-8b6d-6ebc8bfddde6 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:14.870716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 756/1514] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:14.897965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:14.923540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7627ef-1316-452a-a107-19db4c9d0635 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:14.928019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece641e-afce-4951-811c-154fb74ac49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:14.928788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece641e-afce-4951-811c-154fb74ac49c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:14.929094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece641e-afce-4951-811c-154fb74ac49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:14.930091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece641e-afce-4951-811c-154fb74ac49c admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:14.931192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece641e-afce-4951-811c-154fb74ac49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:14.935458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece641e-afce-4951-811c-154fb74ac49c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:15.003971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 759/1515] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:14 2022] POST /identity/v3/projects => generated 383 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:15.005194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56ce28-d77e-412f-8b6d-6ebc8bfddde6 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:15.018285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56ce28-d77e-412f-8b6d-6ebc8bfddde6 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:15.094460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329ec9c-052b-4702-866c-93d4e442df39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:15.095093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ec9c-052b-4702-866c-93d4e442df39 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:15.095372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ec9c-052b-4702-866c-93d4e442df39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:15.095643 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ec9c-052b-4702-866c-93d4e442df39 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:15.096280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329ec9c-052b-4702-866c-93d4e442df39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:15.098299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329ec9c-052b-4702-866c-93d4e442df39 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:15.131532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 757/1516] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:15 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:15.134025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece641e-afce-4951-811c-154fb74ac49c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:15.158197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece641e-afce-4951-811c-154fb74ac49c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:15.158866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092de8d-94c6-4935-bfdd-ec9a2a8136ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:15.158866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092de8d-94c6-4935-bfdd-ec9a2a8136ef admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:15.159220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092de8d-94c6-4935-bfdd-ec9a2a8136ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:15.159431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092de8d-94c6-4935-bfdd-ec9a2a8136ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:15.160437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092de8d-94c6-4935-bfdd-ec9a2a8136ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:15.162103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092de8d-94c6-4935-bfdd-ec9a2a8136ef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:15.179545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 760/1517] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:15.180956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329ec9c-052b-4702-866c-93d4e442df39 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:15.198666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329ec9c-052b-4702-866c-93d4e442df39 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:15.204316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb14201-2615-4697-97aa-8060c36bde32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:15.205102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb14201-2615-4697-97aa-8060c36bde32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:15.205377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb14201-2615-4697-97aa-8060c36bde32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:15.205649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb14201-2615-4697-97aa-8060c36bde32 admin admin] PATH_INFO: `/v3/projects/2330e1b07f334aae88f5430a0d57b0f2/users/457ce6e778a04a74a38634ef2862e2d0/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:15.225622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb14201-2615-4697-97aa-8060c36bde32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2330e1b07f334aae88f5430a0d57b0f2, user_id=457ce6e778a04a74a38634ef2862e2d0, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:15.229937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb14201-2615-4697-97aa-8060c36bde32 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:15.251504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 758/1518] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:15 2022] PUT /identity/v3/projects/2330e1b07f334aae88f5430a0d57b0f2/users/457ce6e778a04a74a38634ef2862e2d0/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:15.255820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092de8d-94c6-4935-bfdd-ec9a2a8136ef admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:15.266997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092de8d-94c6-4935-bfdd-ec9a2a8136ef admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:15.380035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d9040-62d6-4a3a-b02f-c8c7ed5e3bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:15.381353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9040-62d6-4a3a-b02f-c8c7ed5e3bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:15.381955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9040-62d6-4a3a-b02f-c8c7ed5e3bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:15.382297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9040-62d6-4a3a-b02f-c8c7ed5e3bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:15.383639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d9040-62d6-4a3a-b02f-c8c7ed5e3bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:15.385364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d9040-62d6-4a3a-b02f-c8c7ed5e3bc2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:15.400688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 761/1519] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:15.401510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb14201-2615-4697-97aa-8060c36bde32 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:15.416551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb14201-2615-4697-97aa-8060c36bde32 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:15.428167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536c556-cf85-42b0-9e1d-6a3034e436ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:15.430891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536c556-cf85-42b0-9e1d-6a3034e436ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:15.431468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536c556-cf85-42b0-9e1d-6a3034e436ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:15.431888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536c556-cf85-42b0-9e1d-6a3034e436ea admin admin] PATH_INFO: `/v3/projects/2330e1b07f334aae88f5430a0d57b0f2/users/457ce6e778a04a74a38634ef2862e2d0/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:15.434302 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536c556-cf85-42b0-9e1d-6a3034e436ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2330e1b07f334aae88f5430a0d57b0f2, user_id=457ce6e778a04a74a38634ef2862e2d0, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:15.443645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536c556-cf85-42b0-9e1d-6a3034e436ea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:15.455661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 759/1520] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:15 2022] PUT /identity/v3/projects/2330e1b07f334aae88f5430a0d57b0f2/users/457ce6e778a04a74a38634ef2862e2d0/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:15.466260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db991-8e40-4c7e-8205-f03486d823c3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:15.466664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db991-8e40-4c7e-8205-f03486d823c3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:15.466930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db991-8e40-4c7e-8205-f03486d823c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:15.509757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-533db991-8e40-4c7e-8205-f03486d823c3 None None] MFA Rules not processed for user `457ce6e778a04a74a38634ef2862e2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:15.511061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-533db991-8e40-4c7e-8205-f03486d823c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:15.607695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 762/1521] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:15.618801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536c556-cf85-42b0-9e1d-6a3034e436ea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:15.688656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536c556-cf85-42b0-9e1d-6a3034e436ea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:15.693994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41753b-2eff-4db8-a465-1219a696525c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:15.695623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41753b-2eff-4db8-a465-1219a696525c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:15.696308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41753b-2eff-4db8-a465-1219a696525c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:15.696631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41753b-2eff-4db8-a465-1219a696525c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:15.710147 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41753b-2eff-4db8-a465-1219a696525c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:15.712105 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41753b-2eff-4db8-a465-1219a696525c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:15.722708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 760/1522] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:15.875517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533db991-8e40-4c7e-8205-f03486d823c3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:15.891503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533db991-8e40-4c7e-8205-f03486d823c3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:15.900744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089853f-9953-4f46-8693-526ebe30bab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:15.900744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089853f-9953-4f46-8693-526ebe30bab5 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:15.900744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089853f-9953-4f46-8693-526ebe30bab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:15.900744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089853f-9953-4f46-8693-526ebe30bab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:15.969672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:15.982279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089853f-9953-4f46-8693-526ebe30bab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:15.983931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089853f-9953-4f46-8693-526ebe30bab5 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:15.994361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 763/1523] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:17.389216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41753b-2eff-4db8-a465-1219a696525c service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:17.454367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:17.501581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41753b-2eff-4db8-a465-1219a696525c service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:17.512376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:17.513019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:17.514368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:17.514368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] PATH_INFO: `/v3/users/01c8c33c0d54424e99c23178c862ba64` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:17.515640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c8c33c0d54424e99c23178c862ba64)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:17.517795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:17.618091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c8c33c0d54424e99c23178c862ba64'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:17.621873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c8c33c0d54424e99c23178c862ba64'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:17.632135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c8c33c0d54424e99c23178c862ba64'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:17.657301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c8c33c0d54424e99c23178c862ba64'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:17.660620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c8c33c0d54424e99c23178c862ba64'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:17.691699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 761/1524] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:17 2022] DELETE /identity/v3/users/01c8c33c0d54424e99c23178c862ba64 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:17.697765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258068-0105-4b59-8663-aa1e6a6f37be None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:17.699898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258068-0105-4b59-8663-aa1e6a6f37be None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:17.700424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258068-0105-4b59-8663-aa1e6a6f37be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:17.736387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-56258068-0105-4b59-8663-aa1e6a6f37be None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:17.738430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-56258068-0105-4b59-8663-aa1e6a6f37be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:17.836626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 764/1525] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:17.877817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:17.879463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:18.057863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68b03c-79ce-4780-92b3-4f3b83ed0449 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:18.061806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:18.064095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:18.064827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:18.065427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:18.099254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:18.101130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:18.131887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 762/1526] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:18.142888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56258068-0105-4b59-8663-aa1e6a6f37be None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:18.144726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-56258068-0105-4b59-8663-aa1e6a6f37be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:18.240486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56258068-0105-4b59-8663-aa1e6a6f37be None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:18.246848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:18.247691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:18.248141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:18.248553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:18.251722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:18.348554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:18.353435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:18.368076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 765/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:18.533995 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:18.536852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:18.553037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c8542-076c-41ed-ab37-b6ab6face162 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:18.559666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:18.560016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:18.560291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:18.560558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] PATH_INFO: `/v3/users/4fe615abb63e425f88ac26265311eaa9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:18.566887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe615abb63e425f88ac26265311eaa9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:18.568890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:18.627582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:18.629709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:18.671480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe615abb63e425f88ac26265311eaa9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.673381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe615abb63e425f88ac26265311eaa9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.674862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bbe4b-e822-43e4-9d14-ed4b02f48e13 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:18.678273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:18.683936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:18.684887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:18.685360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:18.686104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] PATH_INFO: `/v3/projects/879c14664c7a42e4a5e133b53990462c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:18.687630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe615abb63e425f88ac26265311eaa9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.688536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=879c14664c7a42e4a5e133b53990462c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:18.692887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:18.711047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe615abb63e425f88ac26265311eaa9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.712487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06d262f6-8269-4648-9c74-3b7d662fdd81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe615abb63e425f88ac26265311eaa9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.729101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 763/1528] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:18 2022] DELETE /identity/v3/users/4fe615abb63e425f88ac26265311eaa9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:18.733371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9a49-e139-4cff-9892-2d050eaf5867 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:18.734645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9a49-e139-4cff-9892-2d050eaf5867 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:18.734928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9a49-e139-4cff-9892-2d050eaf5867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:18.772320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:18.774821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-17ed9a49-e139-4cff-9892-2d050eaf5867 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:18.776887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-17ed9a49-e139-4cff-9892-2d050eaf5867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:18.786566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879c14664c7a42e4a5e133b53990462c', 'request_id': 'req-243eccf9-9357-4af7-9765-f0c1c04ecce7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.809380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879c14664c7a42e4a5e133b53990462c', 'request_id': 'req-243eccf9-9357-4af7-9765-f0c1c04ecce7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.823262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879c14664c7a42e4a5e133b53990462c', 'request_id': 'req-243eccf9-9357-4af7-9765-f0c1c04ecce7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.847833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] The token cache is being invalidate because project 879c14664c7a42e4a5e133b53990462c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:18.848136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.849903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:18.855819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 766/1529] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:18 2022] DELETE /identity/v3/projects/879c14664c7a42e4a5e133b53990462c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:18.856231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:18.858758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:18.900184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 764/1530] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:18.901126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed9a49-e139-4cff-9892-2d050eaf5867 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:18.905652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-17ed9a49-e139-4cff-9892-2d050eaf5867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:18.930146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed9a49-e139-4cff-9892-2d050eaf5867 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:18.938072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:18.938758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:18.939030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:18.939300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] PATH_INFO: `/v3/users/c1c10f68800a4adba55338ffbf8ac71f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:18.941095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c10f68800a4adba55338ffbf8ac71f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:18.943504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:19.014417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243eccf9-9357-4af7-9765-f0c1c04ecce7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:19.020081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c10f68800a4adba55338ffbf8ac71f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.020999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c10f68800a4adba55338ffbf8ac71f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.022890 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:19.024188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:19.025582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:19.026527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] PATH_INFO: `/v3/users/3062acca43534df6a3d17af1377e18b1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:19.028445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3062acca43534df6a3d17af1377e18b1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:19.030816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:19.045146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c10f68800a4adba55338ffbf8ac71f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.085822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c10f68800a4adba55338ffbf8ac71f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.088186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c10f68800a4adba55338ffbf8ac71f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.107898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 765/1531] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:18 2022] DELETE /identity/v3/users/c1c10f68800a4adba55338ffbf8ac71f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:19.108738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:19.125484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:19.142515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3062acca43534df6a3d17af1377e18b1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.143263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3062acca43534df6a3d17af1377e18b1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.163425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3062acca43534df6a3d17af1377e18b1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.181526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3062acca43534df6a3d17af1377e18b1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.182188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-aa2b461b-e058-489c-b253-95eb967b5e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3062acca43534df6a3d17af1377e18b1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.214098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 767/1532] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:18 2022] DELETE /identity/v3/users/3062acca43534df6a3d17af1377e18b1 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:19.215892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbeaae-ccac-45a5-95c9-a9e87705b75b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:19.216178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbeaae-ccac-45a5-95c9-a9e87705b75b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:19.216450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbeaae-ccac-45a5-95c9-a9e87705b75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:19.229848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:19.250550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6dfbeaae-ccac-45a5-95c9-a9e87705b75b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:19.250672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6dfbeaae-ccac-45a5-95c9-a9e87705b75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:19.265695 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2979532-f7ac-4f61-b0b5-e4131d3ebc22 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:19.334952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:19.357649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:19.357649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:19.357649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:19.357649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:19.366356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:19.366550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 768/1533] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:19.375393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbeaae-ccac-45a5-95c9-a9e87705b75b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:19.377547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6dfbeaae-ccac-45a5-95c9-a9e87705b75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:19.386725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:19.388378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:19.395191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbeaae-ccac-45a5-95c9-a9e87705b75b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:19.399999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:19.400879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:19.401516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:19.401516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:19.401704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 766/1534] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:19.414252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:19.417454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:19.433606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 769/1535] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:19.761849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:19.763469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:19.791031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afa87e-60cc-4de2-bc78-6ad750ab7fd5 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:19.821357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:19.822015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:19.822384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:19.822685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] PATH_INFO: `/v3/projects/13a60d74e5864745b34d452ad38c1503` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:19.824400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a60d74e5864745b34d452ad38c1503)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:19.828600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:19.913358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a60d74e5864745b34d452ad38c1503', 'request_id': 'req-ec3aa817-166e-4d86-b239-3dea5dca62bb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.921601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:19.923152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:19.930580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a60d74e5864745b34d452ad38c1503', 'request_id': 'req-ec3aa817-166e-4d86-b239-3dea5dca62bb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.937826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a60d74e5864745b34d452ad38c1503', 'request_id': 'req-ec3aa817-166e-4d86-b239-3dea5dca62bb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.956885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] The token cache is being invalidate because project 13a60d74e5864745b34d452ad38c1503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:19.957361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.958466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec3aa817-166e-4d86-b239-3dea5dca62bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:19.960292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 767/1536] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:19 2022] DELETE /identity/v3/projects/13a60d74e5864745b34d452ad38c1503 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:19.964505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bbe99-7aa8-4262-b178-76d74d4589d9 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:19.966016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed200-41ce-48dd-a8dc-1489b52a4ef1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:19.966428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed200-41ce-48dd-a8dc-1489b52a4ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:19.966707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed200-41ce-48dd-a8dc-1489b52a4ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.044696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:20.045351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.045633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.045906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] PATH_INFO: `/v3/projects/613caad261024870aac96b9d28a90bac` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.047245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=613caad261024870aac96b9d28a90bac)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:20.049395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:20.148059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613caad261024870aac96b9d28a90bac', 'request_id': 'req-cedc9346-1576-4162-944d-bee9099f2108'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.148059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613caad261024870aac96b9d28a90bac', 'request_id': 'req-cedc9346-1576-4162-944d-bee9099f2108'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.154361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613caad261024870aac96b9d28a90bac', 'request_id': 'req-cedc9346-1576-4162-944d-bee9099f2108'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.171005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c41ed200-41ce-48dd-a8dc-1489b52a4ef1 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:20.172300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c41ed200-41ce-48dd-a8dc-1489b52a4ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:20.174411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] The token cache is being invalidate because project 613caad261024870aac96b9d28a90bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:20.175074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.176015 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.181120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 770/1537] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:19 2022] DELETE /identity/v3/projects/613caad261024870aac96b9d28a90bac => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:20.181120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:20.181120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:20.286730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 768/1538] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:20.288932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ed200-41ce-48dd-a8dc-1489b52a4ef1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:20.290586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c41ed200-41ce-48dd-a8dc-1489b52a4ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:20.303888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ed200-41ce-48dd-a8dc-1489b52a4ef1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:20.306756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc9346-1576-4162-944d-bee9099f2108 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:20.311251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:20.311922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.312199 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.312470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] PATH_INFO: `/v3/users/c8a107be45834413a4a31c6dbd230307` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.313580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a107be45834413a4a31c6dbd230307)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:20.314527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:20.315406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.315512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:20.315994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.317096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] PATH_INFO: `/v3/projects/5f827170fc784bd493d95361d6401d7f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.318478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f827170fc784bd493d95361d6401d7f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:20.321418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:20.329539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:20.336686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:20.375898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a107be45834413a4a31c6dbd230307'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.376759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a107be45834413a4a31c6dbd230307'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.380746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f827170fc784bd493d95361d6401d7f', 'request_id': 'req-43dffc61-3e4b-4b62-83d1-dfc59f77f286'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.386392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a107be45834413a4a31c6dbd230307'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.387918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f827170fc784bd493d95361d6401d7f', 'request_id': 'req-43dffc61-3e4b-4b62-83d1-dfc59f77f286'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.398662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f827170fc784bd493d95361d6401d7f', 'request_id': 'req-43dffc61-3e4b-4b62-83d1-dfc59f77f286'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.405591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a107be45834413a4a31c6dbd230307'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.406416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a107be45834413a4a31c6dbd230307'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.413180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] The token cache is being invalidate because project 5f827170fc784bd493d95361d6401d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:20.413836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.415254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-43dffc61-3e4b-4b62-83d1-dfc59f77f286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.419640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 771/1539] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:20 2022] DELETE /identity/v3/projects/5f827170fc784bd493d95361d6401d7f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:20.419640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678c5e1-5d26-4ade-b8e7-8eda5dbf18d2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.419640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678c5e1-5d26-4ade-b8e7-8eda5dbf18d2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.419918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 769/1540] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:20 2022] DELETE /identity/v3/users/c8a107be45834413a4a31c6dbd230307 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:20.420944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:20.422370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:20.422554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678c5e1-5d26-4ade-b8e7-8eda5dbf18d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.445558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8678c5e1-5d26-4ade-b8e7-8eda5dbf18d2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:20.446827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8678c5e1-5d26-4ade-b8e7-8eda5dbf18d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:20.524973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1097d65-1fd0-4529-a482-e03b519c7092 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:20.532268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:20.533021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.533386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.533758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] PATH_INFO: `/v3/users/457ce6e778a04a74a38634ef2862e2d0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.536429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] RBAC: Authorizing `identity:delete_user(user_id=457ce6e778a04a74a38634ef2862e2d0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:20.539466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:20.544592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 772/1541] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:20.550305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678c5e1-5d26-4ade-b8e7-8eda5dbf18d2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:20.571175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678c5e1-5d26-4ade-b8e7-8eda5dbf18d2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:20.579467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:20.580353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.580761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.581248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.582388 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:20.582790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:20.584913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:20.605285 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 773/1542] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:25:20.606066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457ce6e778a04a74a38634ef2862e2d0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.606384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:20.607160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457ce6e778a04a74a38634ef2862e2d0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.608417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:20.620090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457ce6e778a04a74a38634ef2862e2d0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.638894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a282301-4b56-4370-a07c-8fa277f36ccf admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:20.649402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:20.651469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.652086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.652632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.653634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:20.656361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:20.673630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457ce6e778a04a74a38634ef2862e2d0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.678309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457ce6e778a04a74a38634ef2862e2d0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:20.703532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 770/1543] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:20 2022] DELETE /identity/v3/users/457ce6e778a04a74a38634ef2862e2d0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:20.705420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 774/1544] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:20 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:20.709777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:20.725330 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:20.795570 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d4f9-dfa4-407d-86f8-f35c0557581b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:20.896834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7c786-de38-4059-b9cb-a62b2cb8a368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:20.898233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:20.900028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:20.903806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7c786-de38-4059-b9cb-a62b2cb8a368 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.904108 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7c786-de38-4059-b9cb-a62b2cb8a368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.904383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7c786-de38-4059-b9cb-a62b2cb8a368 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.911702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7c786-de38-4059-b9cb-a62b2cb8a368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:20.917080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7c786-de38-4059-b9cb-a62b2cb8a368 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:20.921727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5e2ce-b2fa-408c-913b-24c0d3efe9e0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:20.940271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:20.940910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:20.941193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:20.941462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] PATH_INFO: `/v3/projects/53d5bf9f77304c168e84b47978aac30c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:20.943891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d5bf9f77304c168e84b47978aac30c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:20.949172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:20.975125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 771/1545] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:20 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:20.979482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7c786-de38-4059-b9cb-a62b2cb8a368 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:21.007590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7c786-de38-4059-b9cb-a62b2cb8a368 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:21.018085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa46224-5b74-456e-b140-b5c6255be5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:21.018980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa46224-5b74-456e-b140-b5c6255be5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.019110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa46224-5b74-456e-b140-b5c6255be5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.020419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa46224-5b74-456e-b140-b5c6255be5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.021229 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa46224-5b74-456e-b140-b5c6255be5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:21.024352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa46224-5b74-456e-b140-b5c6255be5b2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:21.057204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 772/1546] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:21.060361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa46224-5b74-456e-b140-b5c6255be5b2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:21.070993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d5bf9f77304c168e84b47978aac30c', 'request_id': 'req-1ca6735d-e715-4b98-9731-2c03df58d4e2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.082391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa46224-5b74-456e-b140-b5c6255be5b2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:21.084182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d5bf9f77304c168e84b47978aac30c', 'request_id': 'req-1ca6735d-e715-4b98-9731-2c03df58d4e2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.087716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:21.103210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d5bf9f77304c168e84b47978aac30c', 'request_id': 'req-1ca6735d-e715-4b98-9731-2c03df58d4e2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.121860 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] The token cache is being invalidate because project 53d5bf9f77304c168e84b47978aac30c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:21.122896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.124080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ca6735d-e715-4b98-9731-2c03df58d4e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.130249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 775/1547] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:20 2022] DELETE /identity/v3/projects/53d5bf9f77304c168e84b47978aac30c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:21.132158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a46922-c3b6-4ab5-9171-e356092ac456 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.132499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a46922-c3b6-4ab5-9171-e356092ac456 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.132777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a46922-c3b6-4ab5-9171-e356092ac456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.188053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-33a46922-c3b6-4ab5-9171-e356092ac456 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:21.191514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-33a46922-c3b6-4ab5-9171-e356092ac456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.224465 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:21.228445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.229218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.230198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.231114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:21.238205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:21.267666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 773/1548] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:21.268774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:21.274781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.313841 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129d064-b9c8-4565-b237-88319f7d13b5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:21.330718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:21.331545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.332357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.332642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] PATH_INFO: `/v3/projects/2330e1b07f334aae88f5430a0d57b0f2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.337004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2330e1b07f334aae88f5430a0d57b0f2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:21.341383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:21.354539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 776/1549] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:21.355609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a46922-c3b6-4ab5-9171-e356092ac456 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:21.357150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-33a46922-c3b6-4ab5-9171-e356092ac456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.374185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a46922-c3b6-4ab5-9171-e356092ac456 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:21.396084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632eb2fa-384d-4988-8499-e7c260c6cd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:21.398046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eb2fa-384d-4988-8499-e7c260c6cd40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.399285 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eb2fa-384d-4988-8499-e7c260c6cd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.401595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eb2fa-384d-4988-8499-e7c260c6cd40 admin admin] PATH_INFO: `/v3/projects/efc04e251d344915a5460b91903dc811/users/516cb672c22d48328c2fe5f1ffd3d96d/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.426414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632eb2fa-384d-4988-8499-e7c260c6cd40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc04e251d344915a5460b91903dc811, user_id=516cb672c22d48328c2fe5f1ffd3d96d, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:21.431072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632eb2fa-384d-4988-8499-e7c260c6cd40 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:21.437878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:21.464210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2330e1b07f334aae88f5430a0d57b0f2', 'request_id': 'req-b596e768-b471-45cc-8244-b268125c45c7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.473156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2330e1b07f334aae88f5430a0d57b0f2', 'request_id': 'req-b596e768-b471-45cc-8244-b268125c45c7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.473322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 777/1550] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:21 2022] PUT /identity/v3/projects/efc04e251d344915a5460b91903dc811/users/516cb672c22d48328c2fe5f1ffd3d96d/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:21.483502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2330e1b07f334aae88f5430a0d57b0f2', 'request_id': 'req-b596e768-b471-45cc-8244-b268125c45c7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.484949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13624be-e9de-4f9f-87bf-8491ebd777ed None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.486649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13624be-e9de-4f9f-87bf-8491ebd777ed None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.486933 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13624be-e9de-4f9f-87bf-8491ebd777ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.509036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] The token cache is being invalidate because project 2330e1b07f334aae88f5430a0d57b0f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:21.509648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.510688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.512251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a13624be-e9de-4f9f-87bf-8491ebd777ed None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:21.512574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 774/1551] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:21 2022] DELETE /identity/v3/projects/2330e1b07f334aae88f5430a0d57b0f2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:21.513711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:21.515117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.517147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a13624be-e9de-4f9f-87bf-8491ebd777ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.608643 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596e768-b471-45cc-8244-b268125c45c7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:21.624144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 778/1552] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:21.625441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5a96-5572-427b-bc3d-d899faf21162 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.625725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5a96-5572-427b-bc3d-d899faf21162 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.625995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5a96-5572-427b-bc3d-d899faf21162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.627346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:21.628311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.628756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.629164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] PATH_INFO: `/v3/users/20feb5d2ecc644629a46b7dd19cb29dc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.630623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20feb5d2ecc644629a46b7dd19cb29dc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:21.633509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:21.684565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-030e5a96-5572-427b-bc3d-d899faf21162 None None] MFA Rules not processed for user `516cb672c22d48328c2fe5f1ffd3d96d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:21.686802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-030e5a96-5572-427b-bc3d-d899faf21162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.725087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20feb5d2ecc644629a46b7dd19cb29dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.725761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20feb5d2ecc644629a46b7dd19cb29dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.737654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20feb5d2ecc644629a46b7dd19cb29dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.767600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20feb5d2ecc644629a46b7dd19cb29dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.768461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c984e695-1fab-4e61-9b5b-e81c4a8d3115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20feb5d2ecc644629a46b7dd19cb29dc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:21.769120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:21.786230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 775/1553] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:21 2022] DELETE /identity/v3/users/20feb5d2ecc644629a46b7dd19cb29dc => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:21.787546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af294c1-ed8e-45dd-9450-9508445fbf9f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.787860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af294c1-ed8e-45dd-9450-9508445fbf9f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.788161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af294c1-ed8e-45dd-9450-9508445fbf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.802393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 779/1554] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:21.803613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e5a96-5572-427b-bc3d-d899faf21162 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:21.805983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-030e5a96-5572-427b-bc3d-d899faf21162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.814037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4af294c1-ed8e-45dd-9450-9508445fbf9f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:21.815317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4af294c1-ed8e-45dd-9450-9508445fbf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.826575 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e5a96-5572-427b-bc3d-d899faf21162 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:21.855068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:21.899522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:21.900162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.900523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.900711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.901441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:21.901748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:21.903472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:21.916262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 776/1555] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:21.916262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ff375-6794-4422-bb14-c374ae3a09fa None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.916262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ff375-6794-4422-bb14-c374ae3a09fa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.916262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ff375-6794-4422-bb14-c374ae3a09fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.926935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 780/1556] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:21.926935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:21.928661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.934608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c21ff375-6794-4422-bb14-c374ae3a09fa None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:21.935910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c21ff375-6794-4422-bb14-c374ae3a09fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.942960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7c4c-2633-48a6-8f9e-ad3035490e4b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:21.944133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 777/1557] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:21.945326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2331-625c-420c-8017-6aa651289e43 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.945675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2331-625c-420c-8017-6aa651289e43 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.946033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2331-625c-420c-8017-6aa651289e43 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.947623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 778/1558] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:25:21.948652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb739-d5dc-4c71-ae4f-67d9ef622afb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.948932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb739-d5dc-4c71-ae4f-67d9ef622afb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.949798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb739-d5dc-4c71-ae4f-67d9ef622afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.953979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d2f7-9c64-4c8b-8a2d-8717c58611b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:21.954710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d2f7-9c64-4c8b-8a2d-8717c58611b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:21.956570 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d2f7-9c64-4c8b-8a2d-8717c58611b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:21.956570 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d2f7-9c64-4c8b-8a2d-8717c58611b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:21.956570 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64d2f7-9c64-4c8b-8a2d-8717c58611b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:21.970077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64d2f7-9c64-4c8b-8a2d-8717c58611b3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:21.977868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c42bb739-d5dc-4c71-ae4f-67d9ef622afb None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:21.979326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c42bb739-d5dc-4c71-ae4f-67d9ef622afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:21.990781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 779/1559] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:21.990781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bb739-d5dc-4c71-ae4f-67d9ef622afb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.004836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bb739-d5dc-4c71-ae4f-67d9ef622afb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.013964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 781/1560] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:21 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:22.013964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602ddbe-ed7b-4de3-8f25-c4114c63c8fb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.013964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602ddbe-ed7b-4de3-8f25-c4114c63c8fb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.013964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602ddbe-ed7b-4de3-8f25-c4114c63c8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.016185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.016865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.017451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.017735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.018447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.020448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.036303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8602ddbe-ed7b-4de3-8f25-c4114c63c8fb None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:22.037742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8602ddbe-ed7b-4de3-8f25-c4114c63c8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:22.062348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 780/1561] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:21 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:22.063310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.065435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:22.137215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 782/1562] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:25:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:22.149731 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667a2b0-f075-494f-a742-5facae2db7fc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.153309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602ddbe-ed7b-4de3-8f25-c4114c63c8fb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.154259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abb252-dd0d-4fea-a9b1-c0a3f4e7164f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.155093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb252-dd0d-4fea-a9b1-c0a3f4e7164f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.155514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb252-dd0d-4fea-a9b1-c0a3f4e7164f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.155771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb252-dd0d-4fea-a9b1-c0a3f4e7164f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.231855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abb252-dd0d-4fea-a9b1-c0a3f4e7164f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.233196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:22.233333 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abb252-dd0d-4fea-a9b1-c0a3f4e7164f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.246556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 781/1563] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:22.247159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abb252-dd0d-4fea-a9b1-c0a3f4e7164f service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.249900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602ddbe-ed7b-4de3-8f25-c4114c63c8fb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.255234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.255866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.256142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.256374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abb252-dd0d-4fea-a9b1-c0a3f4e7164f service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.256651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.257426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:22.257671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.259281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.261424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.262025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.262423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.262679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.263646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.265351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.273270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 783/1564] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:22.274236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.286732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdb72c-94c7-4654-becf-19f05a46a904 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.293742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 782/1565] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:22 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:22.293742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.293742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:22.296065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938dcc2-e0a3-415f-87e3-3b60b0ad33af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.296689 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938dcc2-e0a3-415f-87e3-3b60b0ad33af admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.296971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938dcc2-e0a3-415f-87e3-3b60b0ad33af admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.297262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938dcc2-e0a3-415f-87e3-3b60b0ad33af admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.297878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938dcc2-e0a3-415f-87e3-3b60b0ad33af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.301461 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938dcc2-e0a3-415f-87e3-3b60b0ad33af admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.341947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 784/1566] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:22 2022] POST /identity/v3/users => generated 287 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:22.341947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938dcc2-e0a3-415f-87e3-3b60b0ad33af admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.372377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938dcc2-e0a3-415f-87e3-3b60b0ad33af admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.380179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59e993-1097-45a2-88c1-0e3178258157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.381174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e993-1097-45a2-88c1-0e3178258157 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.381529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e993-1097-45a2-88c1-0e3178258157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.381900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e993-1097-45a2-88c1-0e3178258157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.383855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59e993-1097-45a2-88c1-0e3178258157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.386289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59e993-1097-45a2-88c1-0e3178258157 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.389652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:22.453727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 785/1567] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:22 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:22.453727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59e993-1097-45a2-88c1-0e3178258157 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.474297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e1af-b3db-49c6-9958-01ba8f18f1f2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.595666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59e993-1097-45a2-88c1-0e3178258157 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.607025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671cdb4-b10a-4694-b717-17e442ec5185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.608562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671cdb4-b10a-4694-b717-17e442ec5185 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.608974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671cdb4-b10a-4694-b717-17e442ec5185 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.609253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671cdb4-b10a-4694-b717-17e442ec5185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.612145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bad1d-1cdb-4094-9f71-b0d7608826db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.612780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bad1d-1cdb-4094-9f71-b0d7608826db admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.613060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bad1d-1cdb-4094-9f71-b0d7608826db admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.613334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bad1d-1cdb-4094-9f71-b0d7608826db admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.613989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bad1d-1cdb-4094-9f71-b0d7608826db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.616529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bad1d-1cdb-4094-9f71-b0d7608826db admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.624720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4671cdb4-b10a-4694-b717-17e442ec5185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.626411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4671cdb4-b10a-4694-b717-17e442ec5185 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.647117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 783/1568] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:22.647197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 786/1569] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:22.655549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671cdb4-b10a-4694-b717-17e442ec5185 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.656926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bad1d-1cdb-4094-9f71-b0d7608826db admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.669462 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671cdb4-b10a-4694-b717-17e442ec5185 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.681368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bad1d-1cdb-4094-9f71-b0d7608826db admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.682403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73f800-96b3-4bb1-bbfa-9bc2deb0b09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.682963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73f800-96b3-4bb1-bbfa-9bc2deb0b09b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.683327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73f800-96b3-4bb1-bbfa-9bc2deb0b09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.683660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73f800-96b3-4bb1-bbfa-9bc2deb0b09b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.684555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73f800-96b3-4bb1-bbfa-9bc2deb0b09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.687078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73f800-96b3-4bb1-bbfa-9bc2deb0b09b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.687803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb36fe-1d59-457b-93a4-d6d9fdb3b2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.689559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb36fe-1d59-457b-93a4-d6d9fdb3b2bc admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.689998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb36fe-1d59-457b-93a4-d6d9fdb3b2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.691547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb36fe-1d59-457b-93a4-d6d9fdb3b2bc admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.693639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb36fe-1d59-457b-93a4-d6d9fdb3b2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.699687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb36fe-1d59-457b-93a4-d6d9fdb3b2bc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.708576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 784/1570] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:22.709420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73f800-96b3-4bb1-bbfa-9bc2deb0b09b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.720855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73f800-96b3-4bb1-bbfa-9bc2deb0b09b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.729602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d970be6-5a9f-42aa-8b4f-0bd49b4acae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.730297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d970be6-5a9f-42aa-8b4f-0bd49b4acae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.730670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d970be6-5a9f-42aa-8b4f-0bd49b4acae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.731032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d970be6-5a9f-42aa-8b4f-0bd49b4acae4 admin admin] PATH_INFO: `/v3/projects/387697963916491b869f36b7b0ada4cf/users/848f048a6e11428b906259d371838864/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.751321 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 787/1571] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:22 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:22.752183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb36fe-1d59-457b-93a4-d6d9fdb3b2bc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.777885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb36fe-1d59-457b-93a4-d6d9fdb3b2bc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.790572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a27860-bcc3-4952-a25d-1e7f231f0d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.791204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a27860-bcc3-4952-a25d-1e7f231f0d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.791482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a27860-bcc3-4952-a25d-1e7f231f0d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.791783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a27860-bcc3-4952-a25d-1e7f231f0d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.793120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a27860-bcc3-4952-a25d-1e7f231f0d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.795200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a27860-bcc3-4952-a25d-1e7f231f0d7a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.806868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d970be6-5a9f-42aa-8b4f-0bd49b4acae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387697963916491b869f36b7b0ada4cf, user_id=848f048a6e11428b906259d371838864, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.809773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d970be6-5a9f-42aa-8b4f-0bd49b4acae4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.819062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 788/1572] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:22.824100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a27860-bcc3-4952-a25d-1e7f231f0d7a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.832102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 785/1573] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:22 2022] PUT /identity/v3/projects/387697963916491b869f36b7b0ada4cf/users/848f048a6e11428b906259d371838864/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:22.833980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d970be6-5a9f-42aa-8b4f-0bd49b4acae4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.846314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a27860-bcc3-4952-a25d-1e7f231f0d7a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.948937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e4746-7552-452e-900a-674cc24da4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.949663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e4746-7552-452e-900a-674cc24da4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.949940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e4746-7552-452e-900a-674cc24da4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.950216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e4746-7552-452e-900a-674cc24da4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.951717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e4746-7552-452e-900a-674cc24da4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.954128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e4746-7552-452e-900a-674cc24da4e4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.966729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 789/1574] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:22.968021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e4746-7552-452e-900a-674cc24da4e4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:22.973263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d970be6-5a9f-42aa-8b4f-0bd49b4acae4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.979813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dc1e8-9163-4bdf-b21e-033e58473c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.980380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e4746-7552-452e-900a-674cc24da4e4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:22.980873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc1e8-9163-4bdf-b21e-033e58473c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.981442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc1e8-9163-4bdf-b21e-033e58473c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.981856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc1e8-9163-4bdf-b21e-033e58473c5e admin admin] PATH_INFO: `/v3/projects/b2dfaff95d564b70aa0cc4335ce61bcc/users/32c75bd7cb054f5180d4a041b12fb68f/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.985482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eca11-ecc3-435d-9f3b-c7b1a5a2fd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:22.986206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eca11-ecc3-435d-9f3b-c7b1a5a2fd69 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:22.986538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eca11-ecc3-435d-9f3b-c7b1a5a2fd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:22.986811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eca11-ecc3-435d-9f3b-c7b1a5a2fd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:22.987474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055eca11-ecc3-435d-9f3b-c7b1a5a2fd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:22.989236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055eca11-ecc3-435d-9f3b-c7b1a5a2fd69 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:22.996065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:23.003463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 790/1575] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:23.004638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eca11-ecc3-435d-9f3b-c7b1a5a2fd69 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:23.009666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dc1e8-9163-4bdf-b21e-033e58473c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2dfaff95d564b70aa0cc4335ce61bcc, user_id=32c75bd7cb054f5180d4a041b12fb68f, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:23.013956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dc1e8-9163-4bdf-b21e-033e58473c5e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:23.017487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eca11-ecc3-435d-9f3b-c7b1a5a2fd69 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:23.023563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba49e0-8433-4348-b392-a0c1f0642c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:23.024188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba49e0-8433-4348-b392-a0c1f0642c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.024556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba49e0-8433-4348-b392-a0c1f0642c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.024822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba49e0-8433-4348-b392-a0c1f0642c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.025531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba49e0-8433-4348-b392-a0c1f0642c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:23.028241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba49e0-8433-4348-b392-a0c1f0642c33 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:23.034216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 786/1576] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:22 2022] PUT /identity/v3/projects/b2dfaff95d564b70aa0cc4335ce61bcc/users/32c75bd7cb054f5180d4a041b12fb68f/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:23.036194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dc1e8-9163-4bdf-b21e-033e58473c5e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:23.044629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 791/1577] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:23.047626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34de56-558b-4618-bf56-e5cb0302b549 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.047930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34de56-558b-4618-bf56-e5cb0302b549 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.048186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34de56-558b-4618-bf56-e5cb0302b549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.064884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:23.096379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3a34de56-558b-4618-bf56-e5cb0302b549 None None] MFA Rules not processed for user `32c75bd7cb054f5180d4a041b12fb68f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:23.097610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3a34de56-558b-4618-bf56-e5cb0302b549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:23.160361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dc1e8-9163-4bdf-b21e-033e58473c5e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:23.165653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97931a12-d626-4bbc-87a7-fc61e082aed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:23.166533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931a12-d626-4bbc-87a7-fc61e082aed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.167034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931a12-d626-4bbc-87a7-fc61e082aed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.167410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931a12-d626-4bbc-87a7-fc61e082aed1 admin admin] PATH_INFO: `/v3/projects/387697963916491b869f36b7b0ada4cf/users/848f048a6e11428b906259d371838864/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.169839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97931a12-d626-4bbc-87a7-fc61e082aed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387697963916491b869f36b7b0ada4cf, user_id=848f048a6e11428b906259d371838864, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:23.173015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97931a12-d626-4bbc-87a7-fc61e082aed1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:23.188779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 787/1578] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:23 2022] PUT /identity/v3/projects/387697963916491b869f36b7b0ada4cf/users/848f048a6e11428b906259d371838864/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:23.190174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97931a12-d626-4bbc-87a7-fc61e082aed1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:23.216025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 792/1579] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:23.218518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87118209-23cc-4ecc-9289-9afd6e1ec539 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.218518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87118209-23cc-4ecc-9289-9afd6e1ec539 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.218518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87118209-23cc-4ecc-9289-9afd6e1ec539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.255815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-87118209-23cc-4ecc-9289-9afd6e1ec539 None None] MFA Rules not processed for user `848f048a6e11428b906259d371838864`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:23.257360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-87118209-23cc-4ecc-9289-9afd6e1ec539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:23.347534 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97931a12-d626-4bbc-87a7-fc61e082aed1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:23.355594 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 793/1580] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:23.359782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:23.359913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87118209-23cc-4ecc-9289-9afd6e1ec539 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:23.359991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.359991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.359991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] PATH_INFO: `/v3/projects/fd45e85ad131466dac236f7504c03cc3/users/6c325b21d32f4f088e94c17a0e432533/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.360164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-87118209-23cc-4ecc-9289-9afd6e1ec539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:23.409974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd45e85ad131466dac236f7504c03cc3, user_id=6c325b21d32f4f088e94c17a0e432533, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:23.413026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:23.449967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 788/1581] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:23 2022] PUT /identity/v3/projects/fd45e85ad131466dac236f7504c03cc3/users/6c325b21d32f4f088e94c17a0e432533/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:23.452519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:23.456254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:23.480792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87118209-23cc-4ecc-9289-9afd6e1ec539 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:23.530175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:23.555513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420a516-4962-4358-a211-6773f51163cb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:23.560168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3b3a9-b82c-4ea3-834f-d827947888d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:23.561606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3b3a9-b82c-4ea3-834f-d827947888d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.562141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3b3a9-b82c-4ea3-834f-d827947888d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.562385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3b3a9-b82c-4ea3-834f-d827947888d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.586416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:23.587180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.587644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.587923 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.635413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:23.699929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3b3a9-b82c-4ea3-834f-d827947888d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:23.701391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3b3a9-b82c-4ea3-834f-d827947888d0 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:23.713066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:23.714512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:23.725362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 789/1582] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:23.729715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c7573-e9ec-43ab-b240-df7039118fa9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.729994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c7573-e9ec-43ab-b240-df7039118fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.732341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c7573-e9ec-43ab-b240-df7039118fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.737555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 794/1583] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:23.738489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:23.740801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:23.828151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-639c7573-e9ec-43ab-b240-df7039118fa9 None None] MFA Rules not processed for user `6c325b21d32f4f088e94c17a0e432533`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:23.829672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-639c7573-e9ec-43ab-b240-df7039118fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:23.940051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 790/1584] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:23.941159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c7573-e9ec-43ab-b240-df7039118fa9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:23.955515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c7573-e9ec-43ab-b240-df7039118fa9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:23.965920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:23.966318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a16d3-603b-4163-a76d-62b577f36e64 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:23.966768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.967042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.967419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] PATH_INFO: `/v3/users/32c75bd7cb054f5180d4a041b12fb68f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.968853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c75bd7cb054f5180d4a041b12fb68f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:23.971988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:23.987171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:23.988714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:23.990430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:23.993929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] PATH_INFO: `/v3/projects/ff88b874f3d0498bb241e5b4deeb32cf` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:23.997342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff88b874f3d0498bb241e5b4deeb32cf)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:24.001779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:24.077416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c75bd7cb054f5180d4a041b12fb68f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.078570 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c75bd7cb054f5180d4a041b12fb68f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.093553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c75bd7cb054f5180d4a041b12fb68f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.128736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c75bd7cb054f5180d4a041b12fb68f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.131102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c75bd7cb054f5180d4a041b12fb68f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.133617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff88b874f3d0498bb241e5b4deeb32cf', 'request_id': 'req-dbc4d62b-7906-461a-9f40-11999bd1cadb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.149786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff88b874f3d0498bb241e5b4deeb32cf', 'request_id': 'req-dbc4d62b-7906-461a-9f40-11999bd1cadb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.167427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff88b874f3d0498bb241e5b4deeb32cf', 'request_id': 'req-dbc4d62b-7906-461a-9f40-11999bd1cadb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.175749 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 791/1585] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:23 2022] DELETE /identity/v3/users/32c75bd7cb054f5180d4a041b12fb68f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:24.176783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:24.178367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:24.185044 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] The token cache is being invalidate because project ff88b874f3d0498bb241e5b4deeb32cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:24.185614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.186499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbc4d62b-7906-461a-9f40-11999bd1cadb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.188478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 795/1586] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:23 2022] DELETE /identity/v3/projects/ff88b874f3d0498bb241e5b4deeb32cf => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:24.190354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c19d-215e-45ba-b05f-f146923edf52 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.190646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c19d-215e-45ba-b05f-f146923edf52 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.190936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c19d-215e-45ba-b05f-f146923edf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.200213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:24.233349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f2f5c19d-215e-45ba-b05f-f146923edf52 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:24.237066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f2f5c19d-215e-45ba-b05f-f146923edf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:24.304977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:24.346455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77032e-934e-4197-b2fd-d3a877f2e3a5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:24.352222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:24.353131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.353383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.353652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.354471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:24.356529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:24.361225 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 796/1587] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:24.362529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5c19d-215e-45ba-b05f-f146923edf52 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:24.364564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f2f5c19d-215e-45ba-b05f-f146923edf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:24.378667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5c19d-215e-45ba-b05f-f146923edf52 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:24.383960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:24.384594 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.384994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.385335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] PATH_INFO: `/v3/users/cc40dc2bb2a643ce8e485a5d6a3932cf` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.386468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc40dc2bb2a643ce8e485a5d6a3932cf)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:24.388396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:24.392461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 792/1588] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:24 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:24.392461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:24.392461 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:24.441563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc40dc2bb2a643ce8e485a5d6a3932cf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.442238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc40dc2bb2a643ce8e485a5d6a3932cf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.453398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a16c12-0316-42db-9a31-51af4eebe0b7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:24.455395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc40dc2bb2a643ce8e485a5d6a3932cf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.457345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:24.457951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.458228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.458532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.460319 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:24.463563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc40dc2bb2a643ce8e485a5d6a3932cf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.464220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc40dc2bb2a643ce8e485a5d6a3932cf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:24.477265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 797/1589] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:24 2022] DELETE /identity/v3/users/cc40dc2bb2a643ce8e485a5d6a3932cf => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:24.478422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:24.479776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:24.533181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:24.535087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:24.552918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 793/1590] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:24.564339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca2d96-126d-4445-9aa1-96cb95883381 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:24.571363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96b06a-17ca-4f22-8e25-ce11f1de5fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:24.571982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b06a-17ca-4f22-8e25-ce11f1de5fef admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.572605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b06a-17ca-4f22-8e25-ce11f1de5fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.572918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b06a-17ca-4f22-8e25-ce11f1de5fef admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.573598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96b06a-17ca-4f22-8e25-ce11f1de5fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:24.575789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96b06a-17ca-4f22-8e25-ce11f1de5fef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:24.597560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 798/1591] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:24 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:24.603058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:24.612614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b52c5-d900-4725-bd52-23acd844e1d9 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:24.619039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8714e5-f4ec-48c7-a282-d656c5143087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:24.619759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8714e5-f4ec-48c7-a282-d656c5143087 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.620156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8714e5-f4ec-48c7-a282-d656c5143087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.620528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8714e5-f4ec-48c7-a282-d656c5143087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.621273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8714e5-f4ec-48c7-a282-d656c5143087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:24.623008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8714e5-f4ec-48c7-a282-d656c5143087 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:24.636693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 794/1592] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:24.638410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96b06a-17ca-4f22-8e25-ce11f1de5fef admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:24.651670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96b06a-17ca-4f22-8e25-ce11f1de5fef admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:24.657382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291abd1-524b-4776-b012-a67439e4b928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:24.658163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291abd1-524b-4776-b012-a67439e4b928 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.658489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291abd1-524b-4776-b012-a67439e4b928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.658779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291abd1-524b-4776-b012-a67439e4b928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.659666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291abd1-524b-4776-b012-a67439e4b928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:24.661358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291abd1-524b-4776-b012-a67439e4b928 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:24.663173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:24.687624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 799/1593] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:24.688441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8714e5-f4ec-48c7-a282-d656c5143087 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:24.699525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8714e5-f4ec-48c7-a282-d656c5143087 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:24.725158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560c79-e018-43d5-a27f-9c92d6108484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:24.726530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560c79-e018-43d5-a27f-9c92d6108484 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.728197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560c79-e018-43d5-a27f-9c92d6108484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.728772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560c79-e018-43d5-a27f-9c92d6108484 admin admin] PATH_INFO: `/v3/projects/0a234b481d6540c881347e9ae8f8d6e6/users/25584574cb61469990d28a48352a4fdb/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.765870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80560c79-e018-43d5-a27f-9c92d6108484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a234b481d6540c881347e9ae8f8d6e6, user_id=25584574cb61469990d28a48352a4fdb, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:24.768985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80560c79-e018-43d5-a27f-9c92d6108484 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:24.815533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 795/1594] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:24 2022] PUT /identity/v3/projects/0a234b481d6540c881347e9ae8f8d6e6/users/25584574cb61469990d28a48352a4fdb/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:24.824653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05b2e-1dd6-48da-81ec-f540ecd43a91 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:24.825203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05b2e-1dd6-48da-81ec-f540ecd43a91 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:24.825484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05b2e-1dd6-48da-81ec-f540ecd43a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:24.871827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-98e05b2e-1dd6-48da-81ec-f540ecd43a91 None None] MFA Rules not processed for user `25584574cb61469990d28a48352a4fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:24.873942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-98e05b2e-1dd6-48da-81ec-f540ecd43a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:24.973954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 800/1595] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:24.978130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560c79-e018-43d5-a27f-9c92d6108484 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.038055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:25.048058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e05b2e-1dd6-48da-81ec-f540ecd43a91 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.049505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-98e05b2e-1dd6-48da-81ec-f540ecd43a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.081482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560c79-e018-43d5-a27f-9c92d6108484 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.086562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.089559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.089741 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.090016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.090708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.093035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.137605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 796/1596] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:24 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:25.138517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.141920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.161923 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e05b2e-1dd6-48da-81ec-f540ecd43a91 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.216306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d0d04-c6ae-40f7-a852-58d156c4aeeb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.221061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.221686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.221994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.222265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] PATH_INFO: `/v3/projects/b2dfaff95d564b70aa0cc4335ce61bcc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.223424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2dfaff95d564b70aa0cc4335ce61bcc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.225323 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.232880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.233571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.233852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.234125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] PATH_INFO: `/v3/projects/b4fec7422d47439daebf8cdb9afdb6c5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.235565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4fec7422d47439daebf8cdb9afdb6c5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.237741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.278846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2dfaff95d564b70aa0cc4335ce61bcc', 'request_id': 'req-bb79afde-4399-470c-bbce-6829def52fa1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.294533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:25.294669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2dfaff95d564b70aa0cc4335ce61bcc', 'request_id': 'req-bb79afde-4399-470c-bbce-6829def52fa1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.294669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2dfaff95d564b70aa0cc4335ce61bcc', 'request_id': 'req-bb79afde-4399-470c-bbce-6829def52fa1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.297241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4fec7422d47439daebf8cdb9afdb6c5', 'request_id': 'req-70b396da-7bfd-441a-9330-4e31311fa6ca'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.303967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4fec7422d47439daebf8cdb9afdb6c5', 'request_id': 'req-70b396da-7bfd-441a-9330-4e31311fa6ca'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.306961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] The token cache is being invalidate because project b2dfaff95d564b70aa0cc4335ce61bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:25.307269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.307953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.309774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 797/1597] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:25 2022] DELETE /identity/v3/projects/b2dfaff95d564b70aa0cc4335ce61bcc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:25.310739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.312019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.314564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4fec7422d47439daebf8cdb9afdb6c5', 'request_id': 'req-70b396da-7bfd-441a-9330-4e31311fa6ca'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.327849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] The token cache is being invalidate because project b4fec7422d47439daebf8cdb9afdb6c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:25.328346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.329471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70b396da-7bfd-441a-9330-4e31311fa6ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.331593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 801/1598] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:25 2022] DELETE /identity/v3/projects/b4fec7422d47439daebf8cdb9afdb6c5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:25.337453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d333d9-ba56-4af5-8ade-ab54533d78b7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.337453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d333d9-ba56-4af5-8ade-ab54533d78b7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.337801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d333d9-ba56-4af5-8ade-ab54533d78b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.363269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-34d333d9-ba56-4af5-8ade-ab54533d78b7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:25.365077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-34d333d9-ba56-4af5-8ade-ab54533d78b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.386161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:25.395345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79afde-4399-470c-bbce-6829def52fa1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.452548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 802/1599] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:25.453624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d333d9-ba56-4af5-8ade-ab54533d78b7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.454968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-34d333d9-ba56-4af5-8ade-ab54533d78b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.469151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d333d9-ba56-4af5-8ade-ab54533d78b7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.473879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.474545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.474823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.475095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] PATH_INFO: `/v3/users/682a30be74ac430d95b8ef1f9a44bd2a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.476548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=682a30be74ac430d95b8ef1f9a44bd2a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.479497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.483591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590c0f2-6758-4825-aefe-581e0f61e221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.484341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590c0f2-6758-4825-aefe-581e0f61e221 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.484690 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590c0f2-6758-4825-aefe-581e0f61e221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.485035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590c0f2-6758-4825-aefe-581e0f61e221 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.485866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590c0f2-6758-4825-aefe-581e0f61e221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.488144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590c0f2-6758-4825-aefe-581e0f61e221 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.517415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 798/1600] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:25 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:25.519320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a984e-75b9-426b-9296-1ec7c15c656d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.519604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a984e-75b9-426b-9296-1ec7c15c656d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.519930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a984e-75b9-426b-9296-1ec7c15c656d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.543670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4b5a984e-75b9-426b-9296-1ec7c15c656d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:25.545241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b5a984e-75b9-426b-9296-1ec7c15c656d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.553991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 799/1601] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:25.555703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a984e-75b9-426b-9296-1ec7c15c656d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.557201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b5a984e-75b9-426b-9296-1ec7c15c656d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.560611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '682a30be74ac430d95b8ef1f9a44bd2a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.561280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '682a30be74ac430d95b8ef1f9a44bd2a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.568796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a984e-75b9-426b-9296-1ec7c15c656d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.574058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.574685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.574959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.575230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.575868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.577440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.579123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '682a30be74ac430d95b8ef1f9a44bd2a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.588579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '682a30be74ac430d95b8ef1f9a44bd2a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.589178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 800/1602] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:25.589903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.591615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.592780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '682a30be74ac430d95b8ef1f9a44bd2a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:25.606927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77337a3c-da08-40d1-8a66-8b843bc7270c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.607809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 803/1603] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:25 2022] DELETE /identity/v3/users/682a30be74ac430d95b8ef1f9a44bd2a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:25.608557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.609974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.676146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:25.706456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.707089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.707791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.708066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.709101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:25.709502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.712198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.713054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9451d4-98f0-4134-bb08-9039170a0347 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.718694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed80d90-0716-4179-95cd-d75e2a20009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.719358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed80d90-0716-4179-95cd-d75e2a20009d admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.719639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed80d90-0716-4179-95cd-d75e2a20009d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.719910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed80d90-0716-4179-95cd-d75e2a20009d admin admin] PATH_INFO: `/v3/projects/82be7f44d5a541e6b7e58e80355c1d34/users/377c623fd07b4cf49567a5ea53f713c6/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.728400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 801/1604] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:25.735211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.738811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed80d90-0716-4179-95cd-d75e2a20009d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82be7f44d5a541e6b7e58e80355c1d34, user_id=377c623fd07b4cf49567a5ea53f713c6, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.742357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed80d90-0716-4179-95cd-d75e2a20009d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.746356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65be57-407b-4016-a697-e6eba996301d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.751860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c589a-55bb-454a-b437-5155e5ba77bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.752455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c589a-55bb-454a-b437-5155e5ba77bb admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.752732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c589a-55bb-454a-b437-5155e5ba77bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.753005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c589a-55bb-454a-b437-5155e5ba77bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.753655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c589a-55bb-454a-b437-5155e5ba77bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.755575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c589a-55bb-454a-b437-5155e5ba77bb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.759447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 804/1605] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:25 2022] PUT /identity/v3/projects/82be7f44d5a541e6b7e58e80355c1d34/users/377c623fd07b4cf49567a5ea53f713c6/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:25.763421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed80d90-0716-4179-95cd-d75e2a20009d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.785663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 802/1606] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:25 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:25.790700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c589a-55bb-454a-b437-5155e5ba77bb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.846216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed80d90-0716-4179-95cd-d75e2a20009d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.890579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:25.926476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c589a-55bb-454a-b437-5155e5ba77bb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:25.935132 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.935686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.935962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.936231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.936874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.938991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.965972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763afc79-c346-436d-96f5-46709d852558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:25.967164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763afc79-c346-436d-96f5-46709d852558 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:25.967613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763afc79-c346-436d-96f5-46709d852558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:25.968302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763afc79-c346-436d-96f5-46709d852558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:25.969145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763afc79-c346-436d-96f5-46709d852558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:25.971223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763afc79-c346-436d-96f5-46709d852558 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:25.987297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 803/1607] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:25 2022] POST /identity/v3/users => generated 280 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:25.988072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:25.990105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 805/1608] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:25.990915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:25.992150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763afc79-c346-436d-96f5-46709d852558 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.005859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94605dbf-fadc-424a-a2e5-c3f415ebeb87 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.010656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763afc79-c346-436d-96f5-46709d852558 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.014567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.015213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.015492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.015765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] PATH_INFO: `/v3/projects/5e1b0bd5323e419a824dfd333445d775` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.016910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1b0bd5323e419a824dfd333445d775)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.018859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.020259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28468988-881d-4582-9b0f-b343dd588fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.022589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28468988-881d-4582-9b0f-b343dd588fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.023190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28468988-881d-4582-9b0f-b343dd588fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.023600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28468988-881d-4582-9b0f-b343dd588fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.024340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28468988-881d-4582-9b0f-b343dd588fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.026166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28468988-881d-4582-9b0f-b343dd588fcc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.040714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 806/1609] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:26.045187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28468988-881d-4582-9b0f-b343dd588fcc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.058515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28468988-881d-4582-9b0f-b343dd588fcc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.063629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8132667-a606-423a-818a-4894678a45fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.064444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8132667-a606-423a-818a-4894678a45fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.065280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8132667-a606-423a-818a-4894678a45fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.065280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8132667-a606-423a-818a-4894678a45fd admin admin] PATH_INFO: `/v3/projects/82be7f44d5a541e6b7e58e80355c1d34/users/377c623fd07b4cf49567a5ea53f713c6/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.067558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8132667-a606-423a-818a-4894678a45fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=82be7f44d5a541e6b7e58e80355c1d34, user_id=377c623fd07b4cf49567a5ea53f713c6, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.071249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8132667-a606-423a-818a-4894678a45fd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.091752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1b0bd5323e419a824dfd333445d775', 'request_id': 'req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.098269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 807/1610] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:26 2022] PUT /identity/v3/projects/82be7f44d5a541e6b7e58e80355c1d34/users/377c623fd07b4cf49567a5ea53f713c6/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:26.099161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8132667-a606-423a-818a-4894678a45fd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.115382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1b0bd5323e419a824dfd333445d775', 'request_id': 'req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.125470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1b0bd5323e419a824dfd333445d775', 'request_id': 'req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.141467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] The token cache is being invalidate because project 5e1b0bd5323e419a824dfd333445d775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:26.142370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.143184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b4090cf-c534-43c2-8c9b-8fa5bf9bbe58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.149819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 804/1611] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:25 2022] DELETE /identity/v3/projects/5e1b0bd5323e419a824dfd333445d775 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:26.149819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931a5d2-57ad-4aea-9d3e-7d84464e075f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.149819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931a5d2-57ad-4aea-9d3e-7d84464e075f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.149819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931a5d2-57ad-4aea-9d3e-7d84464e075f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.174935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:26.188802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8931a5d2-57ad-4aea-9d3e-7d84464e075f None None] MFA Rules not processed for user `377c623fd07b4cf49567a5ea53f713c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:26.190081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8931a5d2-57ad-4aea-9d3e-7d84464e075f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.203485 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8132667-a606-423a-818a-4894678a45fd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.212626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88571f0-481a-4bd7-bed1-64034245a292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.213243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88571f0-481a-4bd7-bed1-64034245a292 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.213519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88571f0-481a-4bd7-bed1-64034245a292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.213783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88571f0-481a-4bd7-bed1-64034245a292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.214464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88571f0-481a-4bd7-bed1-64034245a292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.216249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88571f0-481a-4bd7-bed1-64034245a292 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.226879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 808/1612] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:26.228278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcd0cf-46ef-4a4f-842f-838ddbf9354f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.228580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcd0cf-46ef-4a4f-842f-838ddbf9354f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.228853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcd0cf-46ef-4a4f-842f-838ddbf9354f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.251196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4fbcd0cf-46ef-4a4f-842f-838ddbf9354f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:26.252402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4fbcd0cf-46ef-4a4f-842f-838ddbf9354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.263963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 809/1613] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:26.264944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcd0cf-46ef-4a4f-842f-838ddbf9354f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.266344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4fbcd0cf-46ef-4a4f-842f-838ddbf9354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.278203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcd0cf-46ef-4a4f-842f-838ddbf9354f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.283445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448d0f9-88d6-4769-821b-cec0234d8255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.284418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448d0f9-88d6-4769-821b-cec0234d8255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.284695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448d0f9-88d6-4769-821b-cec0234d8255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.285084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448d0f9-88d6-4769-821b-cec0234d8255 admin admin] PATH_INFO: `/v3/projects/bc8014ecd4ce4db59e8ea32dc4f7b364/users/c47b5a67cf1a469698c78509e96ee248/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.289048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 805/1614] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:26.290408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133bd4-d224-4689-a095-4fcc3254c2ee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.290758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133bd4-d224-4689-a095-4fcc3254c2ee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.291088 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133bd4-d224-4689-a095-4fcc3254c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.312764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7b133bd4-d224-4689-a095-4fcc3254c2ee None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:26.313950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7b133bd4-d224-4689-a095-4fcc3254c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.314585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448d0f9-88d6-4769-821b-cec0234d8255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8014ecd4ce4db59e8ea32dc4f7b364, user_id=c47b5a67cf1a469698c78509e96ee248, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.317270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448d0f9-88d6-4769-821b-cec0234d8255 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.325686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 806/1615] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:26.327150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b133bd4-d224-4689-a095-4fcc3254c2ee None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.328580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7b133bd4-d224-4689-a095-4fcc3254c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.334690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 810/1616] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:26 2022] PUT /identity/v3/projects/bc8014ecd4ce4db59e8ea32dc4f7b364/users/c47b5a67cf1a469698c78509e96ee248/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:26.335661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448d0f9-88d6-4769-821b-cec0234d8255 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.375590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:26.394879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b133bd4-d224-4689-a095-4fcc3254c2ee None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.419590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448d0f9-88d6-4769-821b-cec0234d8255 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.424315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078813a-1ffc-4ce1-90e8-cf8a7bef21fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.424940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078813a-1ffc-4ce1-90e8-cf8a7bef21fb admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.425209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078813a-1ffc-4ce1-90e8-cf8a7bef21fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.425476 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078813a-1ffc-4ce1-90e8-cf8a7bef21fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.426125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078813a-1ffc-4ce1-90e8-cf8a7bef21fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:26.426527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078813a-1ffc-4ce1-90e8-cf8a7bef21fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.428150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078813a-1ffc-4ce1-90e8-cf8a7bef21fb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.441912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 811/1617] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:26.444296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44991-e4ce-45bf-b141-a216dbd6af5c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.445049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44991-e4ce-45bf-b141-a216dbd6af5c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.445646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44991-e4ce-45bf-b141-a216dbd6af5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.451957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dcf49-7e9e-4d54-97cf-c4990ba23fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.452578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dcf49-7e9e-4d54-97cf-c4990ba23fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.452854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dcf49-7e9e-4d54-97cf-c4990ba23fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.453121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dcf49-7e9e-4d54-97cf-c4990ba23fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.481638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5ca44991-e4ce-45bf-b141-a216dbd6af5c None None] MFA Rules not processed for user `c47b5a67cf1a469698c78509e96ee248`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:26.482887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5ca44991-e4ce-45bf-b141-a216dbd6af5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.521302 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dcf49-7e9e-4d54-97cf-c4990ba23fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.523093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dcf49-7e9e-4d54-97cf-c4990ba23fd6 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.533529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 807/1618] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:26.534454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dcf49-7e9e-4d54-97cf-c4990ba23fd6 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.547252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dcf49-7e9e-4d54-97cf-c4990ba23fd6 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.553229 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c82a2-db70-4dc5-947f-96a5b3debfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.553900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c82a2-db70-4dc5-947f-96a5b3debfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.554863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c82a2-db70-4dc5-947f-96a5b3debfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.555119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c82a2-db70-4dc5-947f-96a5b3debfe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.555771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c82a2-db70-4dc5-947f-96a5b3debfe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.557907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c82a2-db70-4dc5-947f-96a5b3debfe8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.583587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 812/1619] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:26.598597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 808/1620] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:26 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:26.600472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44991-e4ce-45bf-b141-a216dbd6af5c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.622663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c82a2-db70-4dc5-947f-96a5b3debfe8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.622823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44991-e4ce-45bf-b141-a216dbd6af5c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.654444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:26.700263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.700698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.701082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.701232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.701865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.703865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.710613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c82a2-db70-4dc5-947f-96a5b3debfe8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.716598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.717327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.717671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.718079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.720643 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.727171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 813/1621] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:26 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:26.728492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.730061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.742208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07701914-f3d5-4298-b12d-190896f8d6ba admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.746352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb9186-b4e0-42e9-b1c8-fe2108c210d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.747001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb9186-b4e0-42e9-b1c8-fe2108c210d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.747279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb9186-b4e0-42e9-b1c8-fe2108c210d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.747546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb9186-b4e0-42e9-b1c8-fe2108c210d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.770087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:26.799240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.800677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.810011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb9186-b4e0-42e9-b1c8-fe2108c210d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.811156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 809/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:26.811626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb9186-b4e0-42e9-b1c8-fe2108c210d9 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.813599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.821692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 814/1623] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:26.824983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafca39-4d32-4a6d-ac31-311602ec0b1c service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.829834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bd188-2acb-47f0-8eb4-6d29d661ecea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.830453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bd188-2acb-47f0-8eb4-6d29d661ecea admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.830746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bd188-2acb-47f0-8eb4-6d29d661ecea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.830992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bd188-2acb-47f0-8eb4-6d29d661ecea admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.831634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bd188-2acb-47f0-8eb4-6d29d661ecea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.833187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bd188-2acb-47f0-8eb4-6d29d661ecea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.837946 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb9186-b4e0-42e9-b1c8-fe2108c210d9 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.848986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb9186-b4e0-42e9-b1c8-fe2108c210d9 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.849637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 810/1624] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:26.850497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bd188-2acb-47f0-8eb4-6d29d661ecea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.855046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.855573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.855846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.856110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] PATH_INFO: `/v3/users/c47b5a67cf1a469698c78509e96ee248` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.861488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bd188-2acb-47f0-8eb4-6d29d661ecea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.862878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c47b5a67cf1a469698c78509e96ee248)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.864949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.872254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6b22-4a9f-417f-8ec6-0d185955c48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.873434 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6b22-4a9f-417f-8ec6-0d185955c48f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.873704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6b22-4a9f-417f-8ec6-0d185955c48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.873873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6b22-4a9f-417f-8ec6-0d185955c48f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.874672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a6b22-4a9f-417f-8ec6-0d185955c48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.876255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a6b22-4a9f-417f-8ec6-0d185955c48f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.893604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 811/1625] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:26.895230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6b22-4a9f-417f-8ec6-0d185955c48f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:26.909258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6b22-4a9f-417f-8ec6-0d185955c48f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:26.914936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa762da-d1f1-42c2-a265-4a2ff16d0999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:26.915168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa762da-d1f1-42c2-a265-4a2ff16d0999 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.915495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa762da-d1f1-42c2-a265-4a2ff16d0999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.915768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa762da-d1f1-42c2-a265-4a2ff16d0999 admin admin] PATH_INFO: `/v3/projects/1091d780f5bc478fb66c2250051b2737/users/8d47edb58cbb45d29be3cef604ef9624/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.925724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c47b5a67cf1a469698c78509e96ee248'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.926454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c47b5a67cf1a469698c78509e96ee248'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.932508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa762da-d1f1-42c2-a265-4a2ff16d0999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1091d780f5bc478fb66c2250051b2737, user_id=8d47edb58cbb45d29be3cef604ef9624, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:26.935260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa762da-d1f1-42c2-a265-4a2ff16d0999 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:26.941427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c47b5a67cf1a469698c78509e96ee248'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.951367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c47b5a67cf1a469698c78509e96ee248'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.952057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7df5e514-40d9-46cb-a7bd-b11810aecd63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c47b5a67cf1a469698c78509e96ee248'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:26.955650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 812/1626] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:26 2022] PUT /identity/v3/projects/1091d780f5bc478fb66c2250051b2737/users/8d47edb58cbb45d29be3cef604ef9624/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:26.967675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b0990-772b-4c0e-a232-87d17d2f0033 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.968193 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b0990-772b-4c0e-a232-87d17d2f0033 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.968692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b0990-772b-4c0e-a232-87d17d2f0033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.970396 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 815/1627] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:26 2022] DELETE /identity/v3/users/c47b5a67cf1a469698c78509e96ee248 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:26.974808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674bc63-1a30-4dea-b565-d63a9d230d34 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:26.975115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674bc63-1a30-4dea-b565-d63a9d230d34 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:26.975395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674bc63-1a30-4dea-b565-d63a9d230d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:26.997520 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f674bc63-1a30-4dea-b565-d63a9d230d34 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:26.999196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f674bc63-1a30-4dea-b565-d63a9d230d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:26.999478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-574b0990-772b-4c0e-a232-87d17d2f0033 None None] MFA Rules not processed for user `8d47edb58cbb45d29be3cef604ef9624`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:27.000717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-574b0990-772b-4c0e-a232-87d17d2f0033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:27.022036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:27.076273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 816/1628] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:27.083180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674bc63-1a30-4dea-b565-d63a9d230d34 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.083698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 813/1629] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:27.087779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f674bc63-1a30-4dea-b565-d63a9d230d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:27.089687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b0990-772b-4c0e-a232-87d17d2f0033 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.091169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-574b0990-772b-4c0e-a232-87d17d2f0033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:27.107401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b0990-772b-4c0e-a232-87d17d2f0033 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.113928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f9a0-5343-4061-a389-7da275a11311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.114753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f9a0-5343-4061-a389-7da275a11311 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.115095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f9a0-5343-4061-a389-7da275a11311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.115395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f9a0-5343-4061-a389-7da275a11311 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.116180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534f9a0-5343-4061-a389-7da275a11311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.118556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534f9a0-5343-4061-a389-7da275a11311 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.154588 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 814/1630] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:27 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:27.157882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f9a0-5343-4061-a389-7da275a11311 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.162525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674bc63-1a30-4dea-b565-d63a9d230d34 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.247611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:27.253867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f9a0-5343-4061-a389-7da275a11311 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.261430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b0c60-fc5f-427a-aecd-32fc537ed0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.261972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237f1f6-6ac1-4b92-a6ac-1eaa558c2275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.262093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b0c60-fc5f-427a-aecd-32fc537ed0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.262308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b0c60-fc5f-427a-aecd-32fc537ed0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.262605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237f1f6-6ac1-4b92-a6ac-1eaa558c2275 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.262713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b0c60-fc5f-427a-aecd-32fc537ed0d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.262877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237f1f6-6ac1-4b92-a6ac-1eaa558c2275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.263141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237f1f6-6ac1-4b92-a6ac-1eaa558c2275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.263290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b0c60-fc5f-427a-aecd-32fc537ed0d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.265975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b0c60-fc5f-427a-aecd-32fc537ed0d8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.294493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 815/1631] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:27 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:27.298436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b0c60-fc5f-427a-aecd-32fc537ed0d8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.299811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237f1f6-6ac1-4b92-a6ac-1eaa558c2275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.301251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237f1f6-6ac1-4b92-a6ac-1eaa558c2275 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.313356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b0c60-fc5f-427a-aecd-32fc537ed0d8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.315065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 817/1632] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:27.321399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0301b-f2fd-4073-ab1e-d363106b6cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.322609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0301b-f2fd-4073-ab1e-d363106b6cae admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.323091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0301b-f2fd-4073-ab1e-d363106b6cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.323599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0301b-f2fd-4073-ab1e-d363106b6cae admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.325087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0301b-f2fd-4073-ab1e-d363106b6cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.327017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0301b-f2fd-4073-ab1e-d363106b6cae admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.341411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 816/1633] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:27.345992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237f1f6-6ac1-4b92-a6ac-1eaa558c2275 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.357610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237f1f6-6ac1-4b92-a6ac-1eaa558c2275 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.362514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d9ba1-f0ad-472d-8a04-a62a45868a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.363192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d9ba1-f0ad-472d-8a04-a62a45868a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.363600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d9ba1-f0ad-472d-8a04-a62a45868a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.363908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d9ba1-f0ad-472d-8a04-a62a45868a1d admin admin] PATH_INFO: `/v3/projects/31c76583597947069db2a32064f5226e/users/f6b6c7482ddd4c0c99a260aab7744fce/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.380056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d9ba1-f0ad-472d-8a04-a62a45868a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c76583597947069db2a32064f5226e, user_id=f6b6c7482ddd4c0c99a260aab7744fce, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.383269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d9ba1-f0ad-472d-8a04-a62a45868a1d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.396211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 818/1634] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:27 2022] PUT /identity/v3/projects/31c76583597947069db2a32064f5226e/users/f6b6c7482ddd4c0c99a260aab7744fce/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:27.400471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0301b-f2fd-4073-ab1e-d363106b6cae admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.410819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0301b-f2fd-4073-ab1e-d363106b6cae admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.473398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b386295-792f-4808-a321-38b505d08272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.473999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386295-792f-4808-a321-38b505d08272 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.474258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386295-792f-4808-a321-38b505d08272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.474547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386295-792f-4808-a321-38b505d08272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.475217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b386295-792f-4808-a321-38b505d08272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.477923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b386295-792f-4808-a321-38b505d08272 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.490922 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 817/1635] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:27.492548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d9ba1-f0ad-472d-8a04-a62a45868a1d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.502523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d9ba1-f0ad-472d-8a04-a62a45868a1d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.508025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.508718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.508996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.509268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] PATH_INFO: `/v3/projects/31c76583597947069db2a32064f5226e/users/f6b6c7482ddd4c0c99a260aab7744fce/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.511500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c76583597947069db2a32064f5226e, user_id=f6b6c7482ddd4c0c99a260aab7744fce, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.514147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.526187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 819/1636] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:27 2022] PUT /identity/v3/projects/31c76583597947069db2a32064f5226e/users/f6b6c7482ddd4c0c99a260aab7744fce/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:27.536998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba2e56-da1c-4f3d-b217-dbe66df8edb8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.537280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba2e56-da1c-4f3d-b217-dbe66df8edb8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.537554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba2e56-da1c-4f3d-b217-dbe66df8edb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.568054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c6ba2e56-da1c-4f3d-b217-dbe66df8edb8 None None] MFA Rules not processed for user `f6b6c7482ddd4c0c99a260aab7744fce`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:27.569262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c6ba2e56-da1c-4f3d-b217-dbe66df8edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:27.642878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 818/1637] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:27.654466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.656163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:27.715887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:27.719530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba2e56-da1c-4f3d-b217-dbe66df8edb8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.720310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ffad1-19b0-4522-955d-4de89259f4ed admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.722665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c6ba2e56-da1c-4f3d-b217-dbe66df8edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:27.725008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1af4bb-9146-4e73-84c3-998332e2ff31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.725854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1af4bb-9146-4e73-84c3-998332e2ff31 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.726214 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1af4bb-9146-4e73-84c3-998332e2ff31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.727096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1af4bb-9146-4e73-84c3-998332e2ff31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.744444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1af4bb-9146-4e73-84c3-998332e2ff31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.747529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1af4bb-9146-4e73-84c3-998332e2ff31 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.763225 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 820/1638] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:27.815845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:27.823556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba2e56-da1c-4f3d-b217-dbe66df8edb8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.829494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:27.830240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.830651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.831013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] PATH_INFO: `/v3/projects/bc8014ecd4ce4db59e8ea32dc4f7b364` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:27.832269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8014ecd4ce4db59e8ea32dc4f7b364)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:27.834476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:27.914903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8014ecd4ce4db59e8ea32dc4f7b364', 'request_id': 'req-c14d077e-8a26-4808-8af2-16cebad1618d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:27.923166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8014ecd4ce4db59e8ea32dc4f7b364', 'request_id': 'req-c14d077e-8a26-4808-8af2-16cebad1618d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:27.930617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8014ecd4ce4db59e8ea32dc4f7b364', 'request_id': 'req-c14d077e-8a26-4808-8af2-16cebad1618d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:27.940096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1af4bb-9146-4e73-84c3-998332e2ff31 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:27.956916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] The token cache is being invalidate because project bc8014ecd4ce4db59e8ea32dc4f7b364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:27.957778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:27.958178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c14d077e-8a26-4808-8af2-16cebad1618d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:27.958867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1af4bb-9146-4e73-84c3-998332e2ff31 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:27.959605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 819/1639] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:27 2022] DELETE /identity/v3/projects/bc8014ecd4ce4db59e8ea32dc4f7b364 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:27.977271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921f8a0-ac22-45da-a1f5-6f5ee744e536 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:27.978016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921f8a0-ac22-45da-a1f5-6f5ee744e536 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:27.978492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921f8a0-ac22-45da-a1f5-6f5ee744e536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.003972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3921f8a0-ac22-45da-a1f5-6f5ee744e536 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:28.005198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3921f8a0-ac22-45da-a1f5-6f5ee744e536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:28.023027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.023768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.024107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.024385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] PATH_INFO: `/v3/users/8d47edb58cbb45d29be3cef604ef9624` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.025499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d47edb58cbb45d29be3cef604ef9624)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.027486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.067354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 820/1640] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:28.075535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c735-1671-4711-afcd-3fcaa46ab49a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.076062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c735-1671-4711-afcd-3fcaa46ab49a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.076333 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c735-1671-4711-afcd-3fcaa46ab49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.080658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d47edb58cbb45d29be3cef604ef9624'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:28.081366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d47edb58cbb45d29be3cef604ef9624'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:28.093614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d47edb58cbb45d29be3cef604ef9624'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:28.097044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8e81c735-1671-4711-afcd-3fcaa46ab49a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:28.098481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8e81c735-1671-4711-afcd-3fcaa46ab49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:28.101675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d47edb58cbb45d29be3cef604ef9624'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:28.102826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8eeb2705-3ec8-4376-be66-f3c021d0bc22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d47edb58cbb45d29be3cef604ef9624'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:28.107436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 821/1641] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:28.112013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81c735-1671-4711-afcd-3fcaa46ab49a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.115504 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 821/1642] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:27 2022] DELETE /identity/v3/users/8d47edb58cbb45d29be3cef604ef9624 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:28.119736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a0c25-fad1-4a94-8792-e206fb72fe1d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.120097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a0c25-fad1-4a94-8792-e206fb72fe1d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.120369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a0c25-fad1-4a94-8792-e206fb72fe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.121440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81c735-1671-4711-afcd-3fcaa46ab49a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.140436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-149a0c25-fad1-4a94-8792-e206fb72fe1d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:28.141629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-149a0c25-fad1-4a94-8792-e206fb72fe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:28.178494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.179271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.179271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.179819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.180355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:28.180652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.182244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.193793 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 822/1643] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:28.199070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.205697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 822/1644] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:28.209014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39961f-53f0-42ed-84a2-de70b0fd7f7b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.214005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af80847-2eb2-43dd-9aeb-fe6a092ff311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.214852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af80847-2eb2-43dd-9aeb-fe6a092ff311 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.215195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af80847-2eb2-43dd-9aeb-fe6a092ff311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.215483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af80847-2eb2-43dd-9aeb-fe6a092ff311 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.215762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a0c25-fad1-4a94-8792-e206fb72fe1d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.216153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af80847-2eb2-43dd-9aeb-fe6a092ff311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.217174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-149a0c25-fad1-4a94-8792-e206fb72fe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:28.218082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af80847-2eb2-43dd-9aeb-fe6a092ff311 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.232020 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:28.235776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:28.254672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 823/1645] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:28 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:28.255478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af80847-2eb2-43dd-9aeb-fe6a092ff311 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.282044 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a0c25-fad1-4a94-8792-e206fb72fe1d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.322033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af80847-2eb2-43dd-9aeb-fe6a092ff311 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.326934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e97609-9efc-4fe4-9420-3a78cd8047a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.327655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97609-9efc-4fe4-9420-3a78cd8047a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.327971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97609-9efc-4fe4-9420-3a78cd8047a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.328298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97609-9efc-4fe4-9420-3a78cd8047a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.328818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e97609-9efc-4fe4-9420-3a78cd8047a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.331230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e97609-9efc-4fe4-9420-3a78cd8047a2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.344383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526250da-a1b6-4847-a2e8-48f2b6eaa4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.345487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526250da-a1b6-4847-a2e8-48f2b6eaa4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.345943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526250da-a1b6-4847-a2e8-48f2b6eaa4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.346572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526250da-a1b6-4847-a2e8-48f2b6eaa4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.355354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 824/1646] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:28 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:28.359190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e97609-9efc-4fe4-9420-3a78cd8047a2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.360165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526250da-a1b6-4847-a2e8-48f2b6eaa4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.361620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526250da-a1b6-4847-a2e8-48f2b6eaa4b1 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.369530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e97609-9efc-4fe4-9420-3a78cd8047a2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.371223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 823/1647] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:28.374404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4302-610c-4e21-b9a2-b62de21195f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.375005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4302-610c-4e21-b9a2-b62de21195f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.375263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4302-610c-4e21-b9a2-b62de21195f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.375531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4302-610c-4e21-b9a2-b62de21195f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.376167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b4302-610c-4e21-b9a2-b62de21195f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.377751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b4302-610c-4e21-b9a2-b62de21195f5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.390363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 825/1648] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:28.392922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526250da-a1b6-4847-a2e8-48f2b6eaa4b1 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.403161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526250da-a1b6-4847-a2e8-48f2b6eaa4b1 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.407990 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a17233-5322-4e88-85e0-2ca0e0716c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.408790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a17233-5322-4e88-85e0-2ca0e0716c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.409130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a17233-5322-4e88-85e0-2ca0e0716c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.409777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a17233-5322-4e88-85e0-2ca0e0716c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.410043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a17233-5322-4e88-85e0-2ca0e0716c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.411764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a17233-5322-4e88-85e0-2ca0e0716c10 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.423104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 824/1649] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:28.427472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4302-610c-4e21-b9a2-b62de21195f5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.443395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4302-610c-4e21-b9a2-b62de21195f5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.453389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756e3ba-40f9-4cb3-b86f-936c66426fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.454216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756e3ba-40f9-4cb3-b86f-936c66426fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.454799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756e3ba-40f9-4cb3-b86f-936c66426fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.455608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756e3ba-40f9-4cb3-b86f-936c66426fc3 admin admin] PATH_INFO: `/v3/projects/4fd919720b964b4481f6fd8a6ba39841/users/86b2aca3df2147ebbfe5e1531d7b747d/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.473799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756e3ba-40f9-4cb3-b86f-936c66426fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd919720b964b4481f6fd8a6ba39841, user_id=86b2aca3df2147ebbfe5e1531d7b747d, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.476717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756e3ba-40f9-4cb3-b86f-936c66426fc3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.495172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 826/1650] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:28 2022] PUT /identity/v3/projects/4fd919720b964b4481f6fd8a6ba39841/users/86b2aca3df2147ebbfe5e1531d7b747d/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:28.505312 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345f68-b5e8-495e-a938-c493ec002fb5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.505601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345f68-b5e8-495e-a938-c493ec002fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.505874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345f68-b5e8-495e-a938-c493ec002fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.546547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-99345f68-b5e8-495e-a938-c493ec002fb5 None None] MFA Rules not processed for user `86b2aca3df2147ebbfe5e1531d7b747d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:28.547334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-99345f68-b5e8-495e-a938-c493ec002fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:28.656326 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 825/1651] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:28.656930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756e3ba-40f9-4cb3-b86f-936c66426fc3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.741014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756e3ba-40f9-4cb3-b86f-936c66426fc3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.747236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.747923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.748796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.748796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.749397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.751599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.804784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 827/1652] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:28 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:28.809383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99345f68-b5e8-495e-a938-c493ec002fb5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.821233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99345f68-b5e8-495e-a938-c493ec002fb5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.845199 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.847726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:28.880872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:28.896231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:28.901897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9d6c-d886-4598-9091-583e43f10fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.903278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9d6c-d886-4598-9091-583e43f10fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.903710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9d6c-d886-4598-9091-583e43f10fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.904239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9d6c-d886-4598-9091-583e43f10fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.905085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9d6c-d886-4598-9091-583e43f10fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.907664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9d6c-d886-4598-9091-583e43f10fa2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.930266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f518-877b-4fe0-8dac-8435b1971937 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.932110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 826/1653] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:28 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:28.937380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9d6c-d886-4598-9091-583e43f10fa2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.937713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.938370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.938678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.938906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] PATH_INFO: `/v3/projects/1091d780f5bc478fb66c2250051b2737` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.940060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1091d780f5bc478fb66c2250051b2737)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.941944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.949256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9d6c-d886-4598-9091-583e43f10fa2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.954145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3972759-2623-4b8f-8bb4-31b8c443a687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.954920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3972759-2623-4b8f-8bb4-31b8c443a687 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.955200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3972759-2623-4b8f-8bb4-31b8c443a687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.955468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3972759-2623-4b8f-8bb4-31b8c443a687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.956105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3972759-2623-4b8f-8bb4-31b8c443a687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.957726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3972759-2623-4b8f-8bb4-31b8c443a687 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:28.969061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 827/1654] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:28.972757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3972759-2623-4b8f-8bb4-31b8c443a687 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:28.984074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3972759-2623-4b8f-8bb4-31b8c443a687 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:28.988967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cfcbd-2957-4296-8b9c-ebfd155dae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:28.989565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cfcbd-2957-4296-8b9c-ebfd155dae6e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:28.989847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cfcbd-2957-4296-8b9c-ebfd155dae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:28.990236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cfcbd-2957-4296-8b9c-ebfd155dae6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:28.991021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cfcbd-2957-4296-8b9c-ebfd155dae6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:28.992768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cfcbd-2957-4296-8b9c-ebfd155dae6e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.003215 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1091d780f5bc478fb66c2250051b2737', 'request_id': 'req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.009646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 828/1655] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:29.011595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cfcbd-2957-4296-8b9c-ebfd155dae6e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.012771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1091d780f5bc478fb66c2250051b2737', 'request_id': 'req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.021527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cfcbd-2957-4296-8b9c-ebfd155dae6e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.023576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1091d780f5bc478fb66c2250051b2737', 'request_id': 'req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.050795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] The token cache is being invalidate because project 1091d780f5bc478fb66c2250051b2737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:29.051139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.051782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.057616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 828/1656] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:28 2022] DELETE /identity/v3/projects/1091d780f5bc478fb66c2250051b2737 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:29.058548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.060412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.101502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d4958-b543-430f-a593-0b3fb37ca398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.102747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4958-b543-430f-a593-0b3fb37ca398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.103023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4958-b543-430f-a593-0b3fb37ca398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.103291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4958-b543-430f-a593-0b3fb37ca398 admin admin] PATH_INFO: `/v3/projects/f2eef3c897bd434ca79f5ee4105de68b/users/6bda6830f54345bc97eeea49cbda656d/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.123765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d4958-b543-430f-a593-0b3fb37ca398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2eef3c897bd434ca79f5ee4105de68b, user_id=6bda6830f54345bc97eeea49cbda656d, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.128550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d4958-b543-430f-a593-0b3fb37ca398 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.147244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 829/1657] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:29 2022] PUT /identity/v3/projects/f2eef3c897bd434ca79f5ee4105de68b/users/6bda6830f54345bc97eeea49cbda656d/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:29.159885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06c8ec-bbe1-436e-83ef-e36ba5f1546f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.160078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05145c-4805-49cd-9bae-d89359e47b39 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.160200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05145c-4805-49cd-9bae-d89359e47b39 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.160464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05145c-4805-49cd-9bae-d89359e47b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.208579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fc05145c-4805-49cd-9bae-d89359e47b39 None None] MFA Rules not processed for user `6bda6830f54345bc97eeea49cbda656d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:29.209905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc05145c-4805-49cd-9bae-d89359e47b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.235323 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.235958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.236639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.236914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] PATH_INFO: `/v3/users/f6b6c7482ddd4c0c99a260aab7744fce` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.238065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b6c7482ddd4c0c99a260aab7744fce)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.240722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.255150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:29.279478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 830/1658] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:29.283641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05145c-4805-49cd-9bae-d89359e47b39 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.284982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc05145c-4805-49cd-9bae-d89359e47b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.292686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b6c7482ddd4c0c99a260aab7744fce'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.293367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b6c7482ddd4c0c99a260aab7744fce'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.298751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05145c-4805-49cd-9bae-d89359e47b39 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.301008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b6c7482ddd4c0c99a260aab7744fce'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.303856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1af764-5d90-4639-899e-a112df30983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.305011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1af764-5d90-4639-899e-a112df30983c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.305381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1af764-5d90-4639-899e-a112df30983c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.305650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1af764-5d90-4639-899e-a112df30983c admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.306307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1af764-5d90-4639-899e-a112df30983c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.308522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1af764-5d90-4639-899e-a112df30983c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.314794 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b6c7482ddd4c0c99a260aab7744fce'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.315511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b6c7482ddd4c0c99a260aab7744fce'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.328862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 829/1659] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:29 2022] DELETE /identity/v3/users/f6b6c7482ddd4c0c99a260aab7744fce => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:29.339545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 831/1660] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:29 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:29.343117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.344550 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.423589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed773c4-3584-4c1e-8e3c-29b0718fbc3b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.481804 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542653f5-0c9e-41b1-a99d-dd0c8e2faea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.482468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542653f5-0c9e-41b1-a99d-dd0c8e2faea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.482837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542653f5-0c9e-41b1-a99d-dd0c8e2faea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.483112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542653f5-0c9e-41b1-a99d-dd0c8e2faea8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.483832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542653f5-0c9e-41b1-a99d-dd0c8e2faea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.485719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542653f5-0c9e-41b1-a99d-dd0c8e2faea8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.508460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 830/1661] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:29 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:29.512527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1af764-5d90-4639-899e-a112df30983c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.522909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1af764-5d90-4639-899e-a112df30983c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.527659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9c01c-bb29-476d-b4d6-1937d704df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.528255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9c01c-bb29-476d-b4d6-1937d704df30 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.528527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9c01c-bb29-476d-b4d6-1937d704df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.528791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9c01c-bb29-476d-b4d6-1937d704df30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.529424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9c01c-bb29-476d-b4d6-1937d704df30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.531089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9c01c-bb29-476d-b4d6-1937d704df30 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.532420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:29.546359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 832/1662] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:29.547886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542653f5-0c9e-41b1-a99d-dd0c8e2faea8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.558469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542653f5-0c9e-41b1-a99d-dd0c8e2faea8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.563038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.563686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.563962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.564234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] PATH_INFO: `/v3/projects/44e0e1b069494f63af9b01a57936ee15/users/61146deb25214c4281dd7dca18c73a23/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.580504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e0e1b069494f63af9b01a57936ee15, user_id=61146deb25214c4281dd7dca18c73a23, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.583178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.596523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 831/1663] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:29 2022] PUT /identity/v3/projects/44e0e1b069494f63af9b01a57936ee15/users/61146deb25214c4281dd7dca18c73a23/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:29.601551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9c01c-bb29-476d-b4d6-1937d704df30 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.612450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9c01c-bb29-476d-b4d6-1937d704df30 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.626394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.628907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.635047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:29.686742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859f979-fd09-460d-9d61-486f18350a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.687343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859f979-fd09-460d-9d61-486f18350a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.687617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859f979-fd09-460d-9d61-486f18350a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.687883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859f979-fd09-460d-9d61-486f18350a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.688523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859f979-fd09-460d-9d61-486f18350a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.690105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859f979-fd09-460d-9d61-486f18350a5c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.700886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 833/1664] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:29.704951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859f979-fd09-460d-9d61-486f18350a5c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.711283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf17aba-48c2-4755-878d-e5d2ebe64790 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.716808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.716943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859f979-fd09-460d-9d61-486f18350a5c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.717444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.717717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.717986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] PATH_INFO: `/v3/projects/31c76583597947069db2a32064f5226e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.719347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c76583597947069db2a32064f5226e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.721287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.723144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf205050-4b03-49f9-8f33-25dfb0840b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.723809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205050-4b03-49f9-8f33-25dfb0840b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.724086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205050-4b03-49f9-8f33-25dfb0840b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.724353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205050-4b03-49f9-8f33-25dfb0840b1f admin admin] PATH_INFO: `/v3/projects/44e0e1b069494f63af9b01a57936ee15/users/61146deb25214c4281dd7dca18c73a23/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.726700 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf205050-4b03-49f9-8f33-25dfb0840b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e0e1b069494f63af9b01a57936ee15, user_id=61146deb25214c4281dd7dca18c73a23, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:29.729312 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf205050-4b03-49f9-8f33-25dfb0840b1f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:29.740016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 834/1665] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:29 2022] PUT /identity/v3/projects/44e0e1b069494f63af9b01a57936ee15/users/61146deb25214c4281dd7dca18c73a23/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:29.751522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7d41-4c52-4395-8121-e7b2fa8d0845 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.751937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7d41-4c52-4395-8121-e7b2fa8d0845 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.752279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7d41-4c52-4395-8121-e7b2fa8d0845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.777442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c76583597947069db2a32064f5226e', 'request_id': 'req-db82e30e-11c6-4a2c-8842-67e3e10620ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.784090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c76583597947069db2a32064f5226e', 'request_id': 'req-db82e30e-11c6-4a2c-8842-67e3e10620ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.787618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1dbc7d41-4c52-4395-8121-e7b2fa8d0845 None None] MFA Rules not processed for user `61146deb25214c4281dd7dca18c73a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:29.789023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1dbc7d41-4c52-4395-8121-e7b2fa8d0845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.791207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c76583597947069db2a32064f5226e', 'request_id': 'req-db82e30e-11c6-4a2c-8842-67e3e10620ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.816074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] The token cache is being invalidate because project 31c76583597947069db2a32064f5226e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:29.816442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.817285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db82e30e-11c6-4a2c-8842-67e3e10620ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:29.820303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 832/1666] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:29 2022] DELETE /identity/v3/projects/31c76583597947069db2a32064f5226e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:29.824970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a176-436b-428e-be36-0a625b60fe3d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.825316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a176-436b-428e-be36-0a625b60fe3d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.825655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a176-436b-428e-be36-0a625b60fe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.850753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-43b5a176-436b-428e-be36-0a625b60fe3d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:29.852802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43b5a176-436b-428e-be36-0a625b60fe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.880046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 835/1667] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:29.893005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7d41-4c52-4395-8121-e7b2fa8d0845 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.895180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1dbc7d41-4c52-4395-8121-e7b2fa8d0845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.941767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 833/1668] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:29.951081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2d4a-912b-4f90-800b-976014cfa74f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.951512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2d4a-912b-4f90-800b-976014cfa74f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.951892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2d4a-912b-4f90-800b-976014cfa74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.975820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-292b2d4a-912b-4f90-800b-976014cfa74f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:29.978006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-292b2d4a-912b-4f90-800b-976014cfa74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:29.986784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7d41-4c52-4395-8121-e7b2fa8d0845 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:29.986999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 834/1669] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:29.990911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b2d4a-912b-4f90-800b-976014cfa74f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:29.991185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:29.991910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:29.992270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:29.992865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:29.998998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:30.000084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b2d4a-912b-4f90-800b-976014cfa74f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:30.004944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:30.005714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.005997 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.006271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.007769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:30.007996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.009725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.019426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:30.022181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 835/1670] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:25:30.027305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:30.037067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec98448-0085-4aac-99c8-77eb213e0089 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:30.042266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c4a2-064a-4065-bbe5-8b88e6ea2c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:30.042969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c4a2-064a-4065-bbe5-8b88e6ea2c82 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.043256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c4a2-064a-4065-bbe5-8b88e6ea2c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.043555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c4a2-064a-4065-bbe5-8b88e6ea2c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.044185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0c4a2-064a-4065-bbe5-8b88e6ea2c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.046135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0c4a2-064a-4065-bbe5-8b88e6ea2c82 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.058283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:30.064955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.066628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.077714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 836/1671] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:30.091057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 836/1672] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:30 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:30.092192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:30.103647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2541f4-ba9f-4b99-b10d-e85c1a595aac service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:30.162902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c123c-5979-4490-818a-bd97fb64fc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:30.163530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c123c-5979-4490-818a-bd97fb64fc1f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.163858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c123c-5979-4490-818a-bd97fb64fc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.164127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c123c-5979-4490-818a-bd97fb64fc1f admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.164772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c123c-5979-4490-818a-bd97fb64fc1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.166736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c123c-5979-4490-818a-bd97fb64fc1f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.192716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 837/1673] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:30 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:30.194002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c4a2-064a-4065-bbe5-8b88e6ea2c82 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:30.220918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c4a2-064a-4065-bbe5-8b88e6ea2c82 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:30.226280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95176fc-74c4-4587-ae47-22816fb96888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:30.227745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95176fc-74c4-4587-ae47-22816fb96888 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.228022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95176fc-74c4-4587-ae47-22816fb96888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.228324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95176fc-74c4-4587-ae47-22816fb96888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.229035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95176fc-74c4-4587-ae47-22816fb96888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.230830 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95176fc-74c4-4587-ae47-22816fb96888 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.255414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 837/1674] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:30.259976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c123c-5979-4490-818a-bd97fb64fc1f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:30.273797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c123c-5979-4490-818a-bd97fb64fc1f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:30.281420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec60b79-d770-4637-8124-b150fc0ac7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:30.282235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec60b79-d770-4637-8124-b150fc0ac7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.282622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec60b79-d770-4637-8124-b150fc0ac7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.282981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec60b79-d770-4637-8124-b150fc0ac7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.283692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec60b79-d770-4637-8124-b150fc0ac7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.285494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec60b79-d770-4637-8124-b150fc0ac7ca admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.299688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 838/1675] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:30.306777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95176fc-74c4-4587-ae47-22816fb96888 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:30.326576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95176fc-74c4-4587-ae47-22816fb96888 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:30.334988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0dc62-d73b-45af-9a12-21d694a9f0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:30.336594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0dc62-d73b-45af-9a12-21d694a9f0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.337331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0dc62-d73b-45af-9a12-21d694a9f0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.338220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0dc62-d73b-45af-9a12-21d694a9f0e5 admin admin] PATH_INFO: `/v3/projects/3e10fe3e9bf1492599b9c780fc86e485/users/3e40259c2ae14af1b12505b797bc4175/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.375917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0dc62-d73b-45af-9a12-21d694a9f0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e10fe3e9bf1492599b9c780fc86e485, user_id=3e40259c2ae14af1b12505b797bc4175, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.379074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0dc62-d73b-45af-9a12-21d694a9f0e5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.409080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 838/1676] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:30 2022] PUT /identity/v3/projects/3e10fe3e9bf1492599b9c780fc86e485/users/3e40259c2ae14af1b12505b797bc4175/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:30.429915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca7e4-59f8-4a31-bc5a-84880b0ba0b5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.430639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca7e4-59f8-4a31-bc5a-84880b0ba0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.431144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca7e4-59f8-4a31-bc5a-84880b0ba0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.500594 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-16fca7e4-59f8-4a31-bc5a-84880b0ba0b5 None None] MFA Rules not processed for user `3e40259c2ae14af1b12505b797bc4175`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:30.502011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-16fca7e4-59f8-4a31-bc5a-84880b0ba0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:30.626536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 839/1677] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:30.639176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0dc62-d73b-45af-9a12-21d694a9f0e5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:30.731346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0dc62-d73b-45af-9a12-21d694a9f0e5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:30.734869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7953c85-e247-446e-be36-fd34e923627c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:30.735466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7953c85-e247-446e-be36-fd34e923627c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.735740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7953c85-e247-446e-be36-fd34e923627c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.736039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7953c85-e247-446e-be36-fd34e923627c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.745187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7953c85-e247-446e-be36-fd34e923627c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.746636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7953c85-e247-446e-be36-fd34e923627c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.755597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 839/1678] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:30.775237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca7e4-59f8-4a31-bc5a-84880b0ba0b5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:30.805965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:30.839920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca7e4-59f8-4a31-bc5a-84880b0ba0b5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:30.844353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:30.844967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.845238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.845507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] PATH_INFO: `/v3/users/3e40259c2ae14af1b12505b797bc4175` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.846647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e40259c2ae14af1b12505b797bc4175)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:30.848541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:30.895193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e40259c2ae14af1b12505b797bc4175'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:30.895822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e40259c2ae14af1b12505b797bc4175'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:30.907852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e40259c2ae14af1b12505b797bc4175'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:30.914913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e40259c2ae14af1b12505b797bc4175'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:30.915649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e40259c2ae14af1b12505b797bc4175'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:30.927673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 840/1679] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:30 2022] DELETE /identity/v3/users/3e40259c2ae14af1b12505b797bc4175 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:30.932945 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafede7-67cd-47e4-a8c4-7f2c29dbc515 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:30.933274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafede7-67cd-47e4-a8c4-7f2c29dbc515 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:30.933565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafede7-67cd-47e4-a8c4-7f2c29dbc515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:30.953355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4bafede7-67cd-47e4-a8c4-7f2c29dbc515 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:30.954686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4bafede7-67cd-47e4-a8c4-7f2c29dbc515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:31.021034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:31.022206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:31.037245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 840/1680] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:31.048730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafede7-67cd-47e4-a8c4-7f2c29dbc515 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:31.050597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4bafede7-67cd-47e4-a8c4-7f2c29dbc515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:31.126357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1a17-ce49-47ee-9feb-36c70c3ca65d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:31.126357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:31.126357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:31.126357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:31.127101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:31.127101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:31.157663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafede7-67cd-47e4-a8c4-7f2c29dbc515 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:31.162013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de38a8b-48bc-4253-a418-4fe5b9467cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:31.162783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de38a8b-48bc-4253-a418-4fe5b9467cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:31.163153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de38a8b-48bc-4253-a418-4fe5b9467cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:31.164168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de38a8b-48bc-4253-a418-4fe5b9467cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:31.176224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de38a8b-48bc-4253-a418-4fe5b9467cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:31.178264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de38a8b-48bc-4253-a418-4fe5b9467cc9 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:31.190788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 841/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:31.212291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:31.215688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:31.217688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:31.231081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 841/1682] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:31.897494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de38a8b-48bc-4253-a418-4fe5b9467cc9 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:31.900338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:31.904228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de38a8b-48bc-4253-a418-4fe5b9467cc9 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:31.909663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abdecd-5399-4bc1-83f6-3c774053ac16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:31.910467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abdecd-5399-4bc1-83f6-3c774053ac16 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:31.910837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abdecd-5399-4bc1-83f6-3c774053ac16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:31.911197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abdecd-5399-4bc1-83f6-3c774053ac16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:31.913190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c1abdecd-5399-4bc1-83f6-3c774053ac16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:31.935193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:31.936543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:31.956131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9c07b-be2c-478c-bff6-fef951bd7a7d service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:31.965261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:31.966188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:31.966589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:31.967604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] PATH_INFO: `/v3/projects/3e10fe3e9bf1492599b9c780fc86e485` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:31.969419 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e10fe3e9bf1492599b9c780fc86e485)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:31.973157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.009959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abdecd-5399-4bc1-83f6-3c774053ac16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.012299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abdecd-5399-4bc1-83f6-3c774053ac16 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.025020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 842/1683] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:32.045408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e10fe3e9bf1492599b9c780fc86e485', 'request_id': 'req-5848f763-9cb6-4970-83c4-7081412c8e1b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:32.054833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e10fe3e9bf1492599b9c780fc86e485', 'request_id': 'req-5848f763-9cb6-4970-83c4-7081412c8e1b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:32.066293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e10fe3e9bf1492599b9c780fc86e485', 'request_id': 'req-5848f763-9cb6-4970-83c4-7081412c8e1b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:32.092826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] The token cache is being invalidate because project 3e10fe3e9bf1492599b9c780fc86e485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:32.093450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:32.094756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5848f763-9cb6-4970-83c4-7081412c8e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:32.096410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 842/1684] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:31 2022] DELETE /identity/v3/projects/3e10fe3e9bf1492599b9c780fc86e485 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:32.104125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7ec7-a4da-47b5-b2ed-4062b84e2c87 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.104640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7ec7-a4da-47b5-b2ed-4062b84e2c87 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.104640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7ec7-a4da-47b5-b2ed-4062b84e2c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.125667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4f6b7ec7-a4da-47b5-b2ed-4062b84e2c87 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:32.127054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f6b7ec7-a4da-47b5-b2ed-4062b84e2c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:32.222402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 843/1685] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:32.230416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b055ff-b871-499b-b62c-d873fba21d1b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.231085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b055ff-b871-499b-b62c-d873fba21d1b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.231085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b055ff-b871-499b-b62c-d873fba21d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.253587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a2b055ff-b871-499b-b62c-d873fba21d1b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:32.254801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2b055ff-b871-499b-b62c-d873fba21d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:32.263658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 843/1686] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:32.267702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7ec7-a4da-47b5-b2ed-4062b84e2c87 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.281433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7ec7-a4da-47b5-b2ed-4062b84e2c87 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.289418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.290144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.290593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.291041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.291975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:32.292518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.295833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.310227 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 844/1687] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:32.315813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b055ff-b871-499b-b62c-d873fba21d1b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.330660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b055ff-b871-499b-b62c-d873fba21d1b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.339307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e680ea-2267-41e7-86d9-ed74e8929b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.341137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e680ea-2267-41e7-86d9-ed74e8929b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.341517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e680ea-2267-41e7-86d9-ed74e8929b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.341886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e680ea-2267-41e7-86d9-ed74e8929b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.345646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e680ea-2267-41e7-86d9-ed74e8929b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.347837 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e680ea-2267-41e7-86d9-ed74e8929b4e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.385895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 844/1688] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:32 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:32.389992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.402828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187b221-fa01-45b6-96b5-63414139584f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.466590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e78bc-bc3a-4a0f-8cfd-658a477092f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.467200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e78bc-bc3a-4a0f-8cfd-658a477092f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.467478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e78bc-bc3a-4a0f-8cfd-658a477092f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.467749 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e78bc-bc3a-4a0f-8cfd-658a477092f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.468400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e78bc-bc3a-4a0f-8cfd-658a477092f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.470315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e78bc-bc3a-4a0f-8cfd-658a477092f6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.497154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 845/1689] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:32 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:32.501414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e680ea-2267-41e7-86d9-ed74e8929b4e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.512649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e680ea-2267-41e7-86d9-ed74e8929b4e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.517249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5795-614b-48d6-bcc1-fba3f9e190a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.517832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5795-614b-48d6-bcc1-fba3f9e190a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.518123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5795-614b-48d6-bcc1-fba3f9e190a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.518420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5795-614b-48d6-bcc1-fba3f9e190a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.519053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f5795-614b-48d6-bcc1-fba3f9e190a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.520653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f5795-614b-48d6-bcc1-fba3f9e190a2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.531615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 845/1690] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:32.535535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e78bc-bc3a-4a0f-8cfd-658a477092f6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.545345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e78bc-bc3a-4a0f-8cfd-658a477092f6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.549927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dd429-050c-4266-993d-8232686086a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.550543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd429-050c-4266-993d-8232686086a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.550818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd429-050c-4266-993d-8232686086a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.551212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd429-050c-4266-993d-8232686086a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.551955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dd429-050c-4266-993d-8232686086a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.553530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dd429-050c-4266-993d-8232686086a5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.564044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 846/1691] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:32.568116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5795-614b-48d6-bcc1-fba3f9e190a2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.578492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5795-614b-48d6-bcc1-fba3f9e190a2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.583081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e1fb7-d3a9-4114-86a0-285441e565fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.583714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e1fb7-d3a9-4114-86a0-285441e565fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.584004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e1fb7-d3a9-4114-86a0-285441e565fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.584257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e1fb7-d3a9-4114-86a0-285441e565fc admin admin] PATH_INFO: `/v3/projects/5106953452e747e6b7a44a0a4f19ebfa/users/25356b2fbf32430ba2a0c6abee07b52f/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.599063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e1fb7-d3a9-4114-86a0-285441e565fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5106953452e747e6b7a44a0a4f19ebfa, user_id=25356b2fbf32430ba2a0c6abee07b52f, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.601709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e1fb7-d3a9-4114-86a0-285441e565fc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.614394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 846/1692] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:32 2022] PUT /identity/v3/projects/5106953452e747e6b7a44a0a4f19ebfa/users/25356b2fbf32430ba2a0c6abee07b52f/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:32.624960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed8a13-2c3e-4b03-8f9c-3da36b806363 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.625241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed8a13-2c3e-4b03-8f9c-3da36b806363 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.625507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed8a13-2c3e-4b03-8f9c-3da36b806363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.641121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:32.658119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-82ed8a13-2c3e-4b03-8f9c-3da36b806363 None None] MFA Rules not processed for user `25356b2fbf32430ba2a0c6abee07b52f`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:32.659284 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82ed8a13-2c3e-4b03-8f9c-3da36b806363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:32.725251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 847/1693] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:32.730170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e1fb7-d3a9-4114-86a0-285441e565fc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.795520 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e1fb7-d3a9-4114-86a0-285441e565fc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.802009 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954cf50-133a-44d5-bc45-73eebf7eb386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.802729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954cf50-133a-44d5-bc45-73eebf7eb386 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.803016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954cf50-133a-44d5-bc45-73eebf7eb386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.803322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954cf50-133a-44d5-bc45-73eebf7eb386 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.804344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954cf50-133a-44d5-bc45-73eebf7eb386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.806557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954cf50-133a-44d5-bc45-73eebf7eb386 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.830259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:32.843006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 847/1694] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:32 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:32.844301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed8a13-2c3e-4b03-8f9c-3da36b806363 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.856102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed8a13-2c3e-4b03-8f9c-3da36b806363 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.923093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46672-4f7e-4f2d-8706-55cb623807fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.923693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46672-4f7e-4f2d-8706-55cb623807fc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.923964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46672-4f7e-4f2d-8706-55cb623807fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.924228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46672-4f7e-4f2d-8706-55cb623807fc admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.924862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46672-4f7e-4f2d-8706-55cb623807fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.926797 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46672-4f7e-4f2d-8706-55cb623807fc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.952987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 848/1695] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:32 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:32.954131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954cf50-133a-44d5-bc45-73eebf7eb386 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.964440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954cf50-133a-44d5-bc45-73eebf7eb386 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:32.969273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef25f-24e4-4704-9628-5a596ad115c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:32.969939 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef25f-24e4-4704-9628-5a596ad115c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:32.970247 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef25f-24e4-4704-9628-5a596ad115c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:32.970554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef25f-24e4-4704-9628-5a596ad115c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:32.971244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ef25f-24e4-4704-9628-5a596ad115c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:32.972934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ef25f-24e4-4704-9628-5a596ad115c2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:32.986618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 848/1696] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:32.988293 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46672-4f7e-4f2d-8706-55cb623807fc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:32.999217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46672-4f7e-4f2d-8706-55cb623807fc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.003752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7998-9529-490a-a137-0149f05f3c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.004331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7998-9529-490a-a137-0149f05f3c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.004602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7998-9529-490a-a137-0149f05f3c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.004863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7998-9529-490a-a137-0149f05f3c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.005495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7998-9529-490a-a137-0149f05f3c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.008252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7998-9529-490a-a137-0149f05f3c46 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.017909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 849/1697] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:33.022316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef25f-24e4-4704-9628-5a596ad115c2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:33.032273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef25f-24e4-4704-9628-5a596ad115c2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.036924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06776e90-9393-47ff-ab2a-53fafc9f1d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.037570 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776e90-9393-47ff-ab2a-53fafc9f1d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.037848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776e90-9393-47ff-ab2a-53fafc9f1d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.038120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776e90-9393-47ff-ab2a-53fafc9f1d7e admin admin] PATH_INFO: `/v3/projects/6e86ea67982643879aa4bb140a416aa7/users/5431496280fb4cd0b1072e860235e1a8/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.052767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06776e90-9393-47ff-ab2a-53fafc9f1d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e86ea67982643879aa4bb140a416aa7, user_id=5431496280fb4cd0b1072e860235e1a8, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.056177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06776e90-9393-47ff-ab2a-53fafc9f1d7e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.070408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 849/1698] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:33 2022] PUT /identity/v3/projects/6e86ea67982643879aa4bb140a416aa7/users/5431496280fb4cd0b1072e860235e1a8/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:33.080744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94320a53-e0cf-4063-a89a-e73805b82e8f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.081151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94320a53-e0cf-4063-a89a-e73805b82e8f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.081490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94320a53-e0cf-4063-a89a-e73805b82e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.110217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-94320a53-e0cf-4063-a89a-e73805b82e8f None None] MFA Rules not processed for user `5431496280fb4cd0b1072e860235e1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:33.111443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94320a53-e0cf-4063-a89a-e73805b82e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:33.174715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 850/1699] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:33.180074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06776e90-9393-47ff-ab2a-53fafc9f1d7e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:33.241114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06776e90-9393-47ff-ab2a-53fafc9f1d7e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.246380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ec47-92b0-4d9b-8ccf-52ec7d8f8c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.247057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ec47-92b0-4d9b-8ccf-52ec7d8f8c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.247335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ec47-92b0-4d9b-8ccf-52ec7d8f8c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.247720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ec47-92b0-4d9b-8ccf-52ec7d8f8c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.248367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637ec47-92b0-4d9b-8ccf-52ec7d8f8c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.250255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637ec47-92b0-4d9b-8ccf-52ec7d8f8c43 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.277260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 850/1700] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:33 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:33.281412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94320a53-e0cf-4063-a89a-e73805b82e8f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:33.288031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:33.293657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94320a53-e0cf-4063-a89a-e73805b82e8f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.351631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f9ac3-f055-41f9-a9c6-971386142ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.352222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9ac3-f055-41f9-a9c6-971386142ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.352751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9ac3-f055-41f9-a9c6-971386142ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.353020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9ac3-f055-41f9-a9c6-971386142ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.353658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f9ac3-f055-41f9-a9c6-971386142ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.355609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f9ac3-f055-41f9-a9c6-971386142ed6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.380793 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 851/1701] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:33 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:33.383041 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ec47-92b0-4d9b-8ccf-52ec7d8f8c43 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:33.393181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ec47-92b0-4d9b-8ccf-52ec7d8f8c43 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.397724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eda73-5453-42dc-a896-0aa614155218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.398313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eda73-5453-42dc-a896-0aa614155218 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.398613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eda73-5453-42dc-a896-0aa614155218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.398877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eda73-5453-42dc-a896-0aa614155218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.399508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eda73-5453-42dc-a896-0aa614155218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.401095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eda73-5453-42dc-a896-0aa614155218 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.411542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 851/1702] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:33.416212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f9ac3-f055-41f9-a9c6-971386142ed6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:33.426131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f9ac3-f055-41f9-a9c6-971386142ed6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.431424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88e3bd-c58f-4267-8035-d05bc9c3e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.432081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88e3bd-c58f-4267-8035-d05bc9c3e562 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.432359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88e3bd-c58f-4267-8035-d05bc9c3e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.432661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88e3bd-c58f-4267-8035-d05bc9c3e562 admin admin] PATH_INFO: `/v3/projects/518cd00e397242a2a04d8fdda6249751/users/72a0d1a94fa1404a90bcf6b6cf613a94/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.447809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88e3bd-c58f-4267-8035-d05bc9c3e562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518cd00e397242a2a04d8fdda6249751, user_id=72a0d1a94fa1404a90bcf6b6cf613a94, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.450555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88e3bd-c58f-4267-8035-d05bc9c3e562 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.465308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 852/1703] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:33 2022] PUT /identity/v3/projects/518cd00e397242a2a04d8fdda6249751/users/72a0d1a94fa1404a90bcf6b6cf613a94/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:33.469369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eda73-5453-42dc-a896-0aa614155218 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:33.480952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eda73-5453-42dc-a896-0aa614155218 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.543865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38c6c1-823e-491b-89df-1170108d1d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.544498 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38c6c1-823e-491b-89df-1170108d1d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.544776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38c6c1-823e-491b-89df-1170108d1d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.545048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38c6c1-823e-491b-89df-1170108d1d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.545690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38c6c1-823e-491b-89df-1170108d1d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.547308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38c6c1-823e-491b-89df-1170108d1d0b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.558740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 852/1704] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:33.563080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88e3bd-c58f-4267-8035-d05bc9c3e562 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:33.573770 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88e3bd-c58f-4267-8035-d05bc9c3e562 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.578443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.579094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.579372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.579647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] PATH_INFO: `/v3/projects/518cd00e397242a2a04d8fdda6249751/users/72a0d1a94fa1404a90bcf6b6cf613a94/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.581628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518cd00e397242a2a04d8fdda6249751, user_id=72a0d1a94fa1404a90bcf6b6cf613a94, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.584290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.594910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 853/1705] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:33 2022] PUT /identity/v3/projects/518cd00e397242a2a04d8fdda6249751/users/72a0d1a94fa1404a90bcf6b6cf613a94/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:33.607032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e7c5-129c-4312-b0b7-db05318bfc55 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.607346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e7c5-129c-4312-b0b7-db05318bfc55 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.607620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e7c5-129c-4312-b0b7-db05318bfc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.636866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bbd3e7c5-129c-4312-b0b7-db05318bfc55 None None] MFA Rules not processed for user `72a0d1a94fa1404a90bcf6b6cf613a94`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:33.638208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bbd3e7c5-129c-4312-b0b7-db05318bfc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:33.666023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:33.714356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 853/1706] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:33.725639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:33.728334 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:33.795830 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6771b-f77a-418e-8ab6-271baa3e09a6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:33.801282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:33.804518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:33.804518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:33.804518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:33.897507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:33.899023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:33.908608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 854/1707] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:34.939975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3e7c5-129c-4312-b0b7-db05318bfc55 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:34.941376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bbd3e7c5-129c-4312-b0b7-db05318bfc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:34.953542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3e7c5-129c-4312-b0b7-db05318bfc55 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:34.957042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:34.957632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:34.957905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:34.958171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:34.959833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:35.022204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:35.024309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:35.033886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 854/1708] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:38.732344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:38.734142 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:38.736359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:38.746309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d90a8d-8a69-4ec7-bd43-e554c3ac8729 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:38.748787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:38.750240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60f2d4-a5f0-4e9b-810b-167aa4da737f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:38.751286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f2d4-a5f0-4e9b-810b-167aa4da737f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:38.751717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f2d4-a5f0-4e9b-810b-167aa4da737f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:38.752220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f2d4-a5f0-4e9b-810b-167aa4da737f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:38.818616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ceb56-f548-48f4-8759-5f20e07918db service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:38.825232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:38.825959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:38.826242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:38.826540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] PATH_INFO: `/v3/users/6c325b21d32f4f088e94c17a0e432533` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:38.827852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c325b21d32f4f088e94c17a0e432533)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:38.829934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:38.837318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60f2d4-a5f0-4e9b-810b-167aa4da737f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:38.839257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60f2d4-a5f0-4e9b-810b-167aa4da737f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:38.852928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 855/1709] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:38.927107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c325b21d32f4f088e94c17a0e432533'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:38.927833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c325b21d32f4f088e94c17a0e432533'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:38.937286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c325b21d32f4f088e94c17a0e432533'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:38.952467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c325b21d32f4f088e94c17a0e432533'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:38.954195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c325b21d32f4f088e94c17a0e432533'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:38.968376 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 855/1710] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:38 2022] DELETE /identity/v3/users/6c325b21d32f4f088e94c17a0e432533 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:38.973313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c321e8a-e90a-4274-9583-34a8cc037ea0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:38.973781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c321e8a-e90a-4274-9583-34a8cc037ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:38.974055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c321e8a-e90a-4274-9583-34a8cc037ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:38.997813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8c321e8a-e90a-4274-9583-34a8cc037ea0 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:38.999043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8c321e8a-e90a-4274-9583-34a8cc037ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:39.059990 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:39.081972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 856/1711] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:39.097460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:39.099344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:39.165986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fe380-fbe9-4dd9-b6a9-73219c3aca3b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:39.169560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:39.170364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:39.170548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:39.170800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:39.181901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:39.183464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:39.195828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 856/1712] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:39.527514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c321e8a-e90a-4274-9583-34a8cc037ea0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:39.528836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8c321e8a-e90a-4274-9583-34a8cc037ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:39.537825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:39.540692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:39.557108 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb83f64-3726-429e-9788-5a95086fdc26 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:39.565063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:39.565775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:39.566081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:39.566501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] PATH_INFO: `/v3/projects/fd45e85ad131466dac236f7504c03cc3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:39.567969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd45e85ad131466dac236f7504c03cc3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:39.570929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:39.607353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c321e8a-e90a-4274-9583-34a8cc037ea0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:39.612922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:39.613648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:39.614024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:39.614427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:39.614716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:39.618559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:39.642873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd45e85ad131466dac236f7504c03cc3', 'request_id': 'req-4372a446-9c7b-487d-a989-deb972d1746f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.653220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd45e85ad131466dac236f7504c03cc3', 'request_id': 'req-4372a446-9c7b-487d-a989-deb972d1746f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.665543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd45e85ad131466dac236f7504c03cc3', 'request_id': 'req-4372a446-9c7b-487d-a989-deb972d1746f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.682786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] The token cache is being invalidate because project fd45e85ad131466dac236f7504c03cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:39.683154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.683950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.686053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 857/1713] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:39 2022] DELETE /identity/v3/projects/fd45e85ad131466dac236f7504c03cc3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:39.689883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:39.691347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:39.702854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:39.704372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:39.728630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 857/1714] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:39.764033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372a446-9c7b-487d-a989-deb972d1746f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:39.770609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:39.771420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:39.771689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:39.771962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] PATH_INFO: `/v3/users/25584574cb61469990d28a48352a4fdb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:39.774022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25584574cb61469990d28a48352a4fdb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:39.776182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:39.845877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25584574cb61469990d28a48352a4fdb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.846578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25584574cb61469990d28a48352a4fdb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.854799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25584574cb61469990d28a48352a4fdb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.869762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25584574cb61469990d28a48352a4fdb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.870569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25584574cb61469990d28a48352a4fdb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:39.883878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 858/1715] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:39 2022] DELETE /identity/v3/users/25584574cb61469990d28a48352a4fdb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:40.184084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:40.201624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4880b-c05c-4818-bc93-57fdff606e5b service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:40.269411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:40.270048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:40.270339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:40.270619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] PATH_INFO: `/v3/projects/0a234b481d6540c881347e9ae8f8d6e6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:40.271739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a234b481d6540c881347e9ae8f8d6e6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:40.273729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:40.320262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:40.323569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a234b481d6540c881347e9ae8f8d6e6', 'request_id': 'req-006d7c09-ec7b-43e9-a665-53900208f840'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.331391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a234b481d6540c881347e9ae8f8d6e6', 'request_id': 'req-006d7c09-ec7b-43e9-a665-53900208f840'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.340261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a234b481d6540c881347e9ae8f8d6e6', 'request_id': 'req-006d7c09-ec7b-43e9-a665-53900208f840'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.351762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] The token cache is being invalidate because project 0a234b481d6540c881347e9ae8f8d6e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:40.352299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.353180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.354881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 858/1716] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:40 2022] DELETE /identity/v3/projects/0a234b481d6540c881347e9ae8f8d6e6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:40.358623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:40.360098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:40.373237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67795510-19a3-4eee-a7f7-ffab7dbd4bd0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:40.405576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:40.431858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:40.432469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:40.432742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:40.433010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] PATH_INFO: `/v3/users/377c623fd07b4cf49567a5ea53f713c6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:40.434101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377c623fd07b4cf49567a5ea53f713c6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:40.436100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:40.481569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377c623fd07b4cf49567a5ea53f713c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.482216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377c623fd07b4cf49567a5ea53f713c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.489475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377c623fd07b4cf49567a5ea53f713c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.501812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377c623fd07b4cf49567a5ea53f713c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.502475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-46331876-5a70-4f23-a534-5a3a4f023da0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377c623fd07b4cf49567a5ea53f713c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:40.516345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 859/1717] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:40 2022] DELETE /identity/v3/users/377c623fd07b4cf49567a5ea53f713c6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:40.923327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:40.924834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:40.938373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d7c09-ec7b-43e9-a665-53900208f840 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:40.998610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:40.999238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:40.999518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:40.999788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] PATH_INFO: `/v3/projects/82be7f44d5a541e6b7e58e80355c1d34` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.000892 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=82be7f44d5a541e6b7e58e80355c1d34)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:41.002827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:41.049834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82be7f44d5a541e6b7e58e80355c1d34', 'request_id': 'req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:41.055987 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82be7f44d5a541e6b7e58e80355c1d34', 'request_id': 'req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:41.063222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82be7f44d5a541e6b7e58e80355c1d34', 'request_id': 'req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:41.075056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] The token cache is being invalidate because project 82be7f44d5a541e6b7e58e80355c1d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:41.075347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:41.076003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8578ea21-ac08-46cc-bc0d-104bb17ec4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:41.077507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 859/1718] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:40 2022] DELETE /identity/v3/projects/82be7f44d5a541e6b7e58e80355c1d34 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:41.083375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6142c-49d3-4cd7-995f-748d496d3c7c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.083837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6142c-49d3-4cd7-995f-748d496d3c7c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.083963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6142c-49d3-4cd7-995f-748d496d3c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.105055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6da6142c-49d3-4cd7-995f-748d496d3c7c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:41.106296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6da6142c-49d3-4cd7-995f-748d496d3c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:41.166699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 860/1719] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:41.175382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58ba6-7aed-440b-afb4-c420cc866b0d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.175749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58ba6-7aed-440b-afb4-c420cc866b0d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.176092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58ba6-7aed-440b-afb4-c420cc866b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.207280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3ff58ba6-7aed-440b-afb4-c420cc866b0d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:41.208602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3ff58ba6-7aed-440b-afb4-c420cc866b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:41.216768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 860/1720] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:41.221327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6142c-49d3-4cd7-995f-748d496d3c7c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:41.231342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6142c-49d3-4cd7-995f-748d496d3c7c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:41.236006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:41.236621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.236895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.237160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.237824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:41.238122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:41.239754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:41.252582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 861/1721] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:25:41.257826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff58ba6-7aed-440b-afb4-c420cc866b0d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:41.267135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff58ba6-7aed-440b-afb4-c420cc866b0d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:41.271725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6927cd0-0542-4d06-a25a-81d9ce59b8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:41.272327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927cd0-0542-4d06-a25a-81d9ce59b8ab admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.272631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927cd0-0542-4d06-a25a-81d9ce59b8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.273296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927cd0-0542-4d06-a25a-81d9ce59b8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.273678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6927cd0-0542-4d06-a25a-81d9ce59b8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:41.275626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6927cd0-0542-4d06-a25a-81d9ce59b8ab admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:41.303459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 861/1722] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:41 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:41.308354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:41.318155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25e93c-2be2-4f43-8425-b4c2a2f2d215 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:41.358747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:41.376047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304485f-9d8b-4c6d-9679-aac4aabbf8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:41.376653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304485f-9d8b-4c6d-9679-aac4aabbf8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.376924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304485f-9d8b-4c6d-9679-aac4aabbf8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.377191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304485f-9d8b-4c6d-9679-aac4aabbf8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.377815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304485f-9d8b-4c6d-9679-aac4aabbf8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:41.379738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304485f-9d8b-4c6d-9679-aac4aabbf8d5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:41.402045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 862/1723] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:41 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:41.406301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6927cd0-0542-4d06-a25a-81d9ce59b8ab admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:41.417824 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6927cd0-0542-4d06-a25a-81d9ce59b8ab admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:41.422673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a6ac-bb8c-4d82-8ba3-0838e3cbae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:41.423273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a6ac-bb8c-4d82-8ba3-0838e3cbae08 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.423550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a6ac-bb8c-4d82-8ba3-0838e3cbae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.423852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a6ac-bb8c-4d82-8ba3-0838e3cbae08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.424547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8a6ac-bb8c-4d82-8ba3-0838e3cbae08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:41.426240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8a6ac-bb8c-4d82-8ba3-0838e3cbae08 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:41.437946 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 862/1724] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:41.441991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304485f-9d8b-4c6d-9679-aac4aabbf8d5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:41.452236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304485f-9d8b-4c6d-9679-aac4aabbf8d5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:41.457356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368987ec-87df-4940-8d46-9957183ad96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:41.457956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368987ec-87df-4940-8d46-9957183ad96e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.458232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368987ec-87df-4940-8d46-9957183ad96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.458590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368987ec-87df-4940-8d46-9957183ad96e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.459792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368987ec-87df-4940-8d46-9957183ad96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:41.462045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368987ec-87df-4940-8d46-9957183ad96e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:41.473167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 863/1725] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:41.479047 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a6ac-bb8c-4d82-8ba3-0838e3cbae08 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:41.489072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a6ac-bb8c-4d82-8ba3-0838e3cbae08 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:41.494718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:41.495373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.495649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.495923 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] PATH_INFO: `/v3/projects/667dc2dbb03d448897690e87efde3898/users/c8e673c433af430faa62d21fa124f0a6/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.511767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=667dc2dbb03d448897690e87efde3898, user_id=c8e673c433af430faa62d21fa124f0a6, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:41.514468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:41.523930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:41.529093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 863/1726] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:41 2022] PUT /identity/v3/projects/667dc2dbb03d448897690e87efde3898/users/c8e673c433af430faa62d21fa124f0a6/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:41.534087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d05bc-0c95-424c-9dc5-f54aa74bee76 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.534398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d05bc-0c95-424c-9dc5-f54aa74bee76 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.534672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d05bc-0c95-424c-9dc5-f54aa74bee76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.553670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-593d05bc-0c95-424c-9dc5-f54aa74bee76 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:41.554916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-593d05bc-0c95-424c-9dc5-f54aa74bee76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:41.613324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 864/1727] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:41.623256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:41.624918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:41.680811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf0411-d6bb-4036-96c2-018e3ac699f1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:41.684471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b82ba-7b59-4aaa-b179-701132e6508c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:41.685069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b82ba-7b59-4aaa-b179-701132e6508c service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:41.685344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b82ba-7b59-4aaa-b179-701132e6508c service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:41.685610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b82ba-7b59-4aaa-b179-701132e6508c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:41.694866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b82ba-7b59-4aaa-b179-701132e6508c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:41.696403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b82ba-7b59-4aaa-b179-701132e6508c service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:41.705685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 864/1728] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:42.459207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe757-642b-4a03-94b3-c69db1aec6fe None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:42.460253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe757-642b-4a03-94b3-c69db1aec6fe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:42.460771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe757-642b-4a03-94b3-c69db1aec6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:42.494796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3babe757-642b-4a03-94b3-c69db1aec6fe None None] MFA Rules not processed for user `c8e673c433af430faa62d21fa124f0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:42.496008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3babe757-642b-4a03-94b3-c69db1aec6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:42.582660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 865/1729] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:42.584609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b82ba-7b59-4aaa-b179-701132e6508c service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:42.598492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b82ba-7b59-4aaa-b179-701132e6508c service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:42.604710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6e26-143b-42a7-89a3-8855f139c6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:42.605335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6e26-143b-42a7-89a3-8855f139c6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:42.605614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6e26-143b-42a7-89a3-8855f139c6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:42.605886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6e26-143b-42a7-89a3-8855f139c6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:42.606606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b6e26-143b-42a7-89a3-8855f139c6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:42.608554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b6e26-143b-42a7-89a3-8855f139c6d2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:42.639706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 865/1730] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:42 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:25:42.644018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe757-642b-4a03-94b3-c69db1aec6fe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:42.655178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe757-642b-4a03-94b3-c69db1aec6fe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:42.705809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:42.717380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e94c2-98c4-4e67-97ce-b86cc0e9a25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:42.718709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e94c2-98c4-4e67-97ce-b86cc0e9a25e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:42.718709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e94c2-98c4-4e67-97ce-b86cc0e9a25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:42.718709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e94c2-98c4-4e67-97ce-b86cc0e9a25e admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:42.719307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e94c2-98c4-4e67-97ce-b86cc0e9a25e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:42.721210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e94c2-98c4-4e67-97ce-b86cc0e9a25e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:42.745420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 866/1731] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:42 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:42.746620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6e26-143b-42a7-89a3-8855f139c6d2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:42.756958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6e26-143b-42a7-89a3-8855f139c6d2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:42.761483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fbcb2-98fe-480a-a618-5571e1046333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:42.762103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fbcb2-98fe-480a-a618-5571e1046333 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:42.762435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fbcb2-98fe-480a-a618-5571e1046333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:42.762708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fbcb2-98fe-480a-a618-5571e1046333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:42.763351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fbcb2-98fe-480a-a618-5571e1046333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:42.765174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fbcb2-98fe-480a-a618-5571e1046333 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:42.776783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 866/1732] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:42.780834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e94c2-98c4-4e67-97ce-b86cc0e9a25e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:42.791487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e94c2-98c4-4e67-97ce-b86cc0e9a25e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:42.796251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f389f75-852c-4438-b0a0-b774c632c73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:42.797053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389f75-852c-4438-b0a0-b774c632c73e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:42.797161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389f75-852c-4438-b0a0-b774c632c73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:42.797417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389f75-852c-4438-b0a0-b774c632c73e admin admin] PATH_INFO: `/v3/projects/2b4d79715f514596a8e128bce2362603/users/e0f70cf3fda94e6dbec0cca4c404553b/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:42.813283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f389f75-852c-4438-b0a0-b774c632c73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4d79715f514596a8e128bce2362603, user_id=e0f70cf3fda94e6dbec0cca4c404553b, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:42.815979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f389f75-852c-4438-b0a0-b774c632c73e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:42.828720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 867/1733] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:42 2022] PUT /identity/v3/projects/2b4d79715f514596a8e128bce2362603/users/e0f70cf3fda94e6dbec0cca4c404553b/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:42.832736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fbcb2-98fe-480a-a618-5571e1046333 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:42.842996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fbcb2-98fe-480a-a618-5571e1046333 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:42.898883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc463fba-28c6-4ac5-8556-25695ca9921e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:42.899412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463fba-28c6-4ac5-8556-25695ca9921e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:42.899686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463fba-28c6-4ac5-8556-25695ca9921e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:42.899955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc463fba-28c6-4ac5-8556-25695ca9921e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:42.900583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc463fba-28c6-4ac5-8556-25695ca9921e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:42.902158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc463fba-28c6-4ac5-8556-25695ca9921e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:42.911729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 867/1734] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:42.916488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f389f75-852c-4438-b0a0-b774c632c73e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:42.926665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f389f75-852c-4438-b0a0-b774c632c73e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:42.931456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:42.932092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:42.932367 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:42.932636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] PATH_INFO: `/v3/projects/2b4d79715f514596a8e128bce2362603/users/e0f70cf3fda94e6dbec0cca4c404553b/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:42.934554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4d79715f514596a8e128bce2362603, user_id=e0f70cf3fda94e6dbec0cca4c404553b, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:42.937180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:42.947254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 868/1735] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:42 2022] PUT /identity/v3/projects/2b4d79715f514596a8e128bce2362603/users/e0f70cf3fda94e6dbec0cca4c404553b/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:43.776929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345dab-e3ae-458b-ba0f-d64d848f5d21 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:43.777410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345dab-e3ae-458b-ba0f-d64d848f5d21 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:43.777510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345dab-e3ae-458b-ba0f-d64d848f5d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:43.808332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-94345dab-e3ae-458b-ba0f-d64d848f5d21 None None] MFA Rules not processed for user `e0f70cf3fda94e6dbec0cca4c404553b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:43.809629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-94345dab-e3ae-458b-ba0f-d64d848f5d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:43.866741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:43.890821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 868/1736] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:43.909644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:43.915786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:43.979040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71b01f-713f-49ba-93bb-2698a1ca6082 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:43.982729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:43.983609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:43.983906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:43.984178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:44.042812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:44.044245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:44.053413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 869/1737] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:46.480665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94345dab-e3ae-458b-ba0f-d64d848f5d21 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:46.490768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94345dab-e3ae-458b-ba0f-d64d848f5d21 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:46.494647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689fc85-40c1-4b6f-be36-0d93683cd953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:46.495330 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689fc85-40c1-4b6f-be36-0d93683cd953 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:46.495677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689fc85-40c1-4b6f-be36-0d93683cd953 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:46.497439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689fc85-40c1-4b6f-be36-0d93683cd953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:46.507474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689fc85-40c1-4b6f-be36-0d93683cd953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:46.509189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689fc85-40c1-4b6f-be36-0d93683cd953 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:46.520865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 869/1738] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:46.682441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:46.684682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:46.797890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:46.805725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab531981-637f-4f80-abbe-f76e447d92b3 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:46.813167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:46.814244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:46.814729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:46.815171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] PATH_INFO: `/v3/users/86b2aca3df2147ebbfe5e1531d7b747d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:46.816691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b2aca3df2147ebbfe5e1531d7b747d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:46.818840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:46.882953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b2aca3df2147ebbfe5e1531d7b747d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:46.883744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b2aca3df2147ebbfe5e1531d7b747d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:46.893531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b2aca3df2147ebbfe5e1531d7b747d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:46.914304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b2aca3df2147ebbfe5e1531d7b747d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:46.915773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b2aca3df2147ebbfe5e1531d7b747d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:46.933197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 870/1739] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:46 2022] DELETE /identity/v3/users/86b2aca3df2147ebbfe5e1531d7b747d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:46.938262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa78d0-49f0-45b5-82ff-12053b3eaf8d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:46.938565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa78d0-49f0-45b5-82ff-12053b3eaf8d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:46.938995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa78d0-49f0-45b5-82ff-12053b3eaf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:46.971379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b6fa78d0-49f0-45b5-82ff-12053b3eaf8d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:46.973012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b6fa78d0-49f0-45b5-82ff-12053b3eaf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:47.048174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 870/1740] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:47.057758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:47.059681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:47.138802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb414d-45db-4620-b981-0be076c667be admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:47.144575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5749623-1251-4b6c-b659-f5051f338eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:47.145255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749623-1251-4b6c-b659-f5051f338eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:47.145602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749623-1251-4b6c-b659-f5051f338eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:47.146247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749623-1251-4b6c-b659-f5051f338eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:47.158439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5749623-1251-4b6c-b659-f5051f338eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:47.160056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5749623-1251-4b6c-b659-f5051f338eb9 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:47.173492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 871/1741] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:47.910275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa78d0-49f0-45b5-82ff-12053b3eaf8d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:47.912001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b6fa78d0-49f0-45b5-82ff-12053b3eaf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:47.926509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa78d0-49f0-45b5-82ff-12053b3eaf8d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:47.934054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:47.935218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:47.935619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:47.935999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] PATH_INFO: `/v3/projects/4fd919720b964b4481f6fd8a6ba39841` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:47.937264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd919720b964b4481f6fd8a6ba39841)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:47.939498 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:48.015835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd919720b964b4481f6fd8a6ba39841', 'request_id': 'req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.025846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd919720b964b4481f6fd8a6ba39841', 'request_id': 'req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.036477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd919720b964b4481f6fd8a6ba39841', 'request_id': 'req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.050872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] The token cache is being invalidate because project 4fd919720b964b4481f6fd8a6ba39841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:48.051251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.052078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.053921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 871/1742] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:47 2022] DELETE /identity/v3/projects/4fd919720b964b4481f6fd8a6ba39841 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:48.058446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5749623-1251-4b6c-b659-f5051f338eb9 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:48.073682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5749623-1251-4b6c-b659-f5051f338eb9 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:48.157420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:48.158205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:48.158588 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:48.158931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] PATH_INFO: `/v3/users/6bda6830f54345bc97eeea49cbda656d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:48.160121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bda6830f54345bc97eeea49cbda656d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:48.162124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:48.185969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:48.211889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bda6830f54345bc97eeea49cbda656d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.212540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bda6830f54345bc97eeea49cbda656d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.219375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bda6830f54345bc97eeea49cbda656d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.231704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bda6830f54345bc97eeea49cbda656d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.232345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bda6830f54345bc97eeea49cbda656d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.245015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 872/1743] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:48 2022] DELETE /identity/v3/users/6bda6830f54345bc97eeea49cbda656d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:48.531748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:48.533549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:48.556212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00d249-d8ca-4b8c-a6ba-b78b9bd9383e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:48.624955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:48.625619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:48.625944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:48.626161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] PATH_INFO: `/v3/projects/f2eef3c897bd434ca79f5ee4105de68b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:48.627333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2eef3c897bd434ca79f5ee4105de68b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:48.629234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:48.632554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:48.682059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2eef3c897bd434ca79f5ee4105de68b', 'request_id': 'req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.688363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2eef3c897bd434ca79f5ee4105de68b', 'request_id': 'req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.697848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2eef3c897bd434ca79f5ee4105de68b', 'request_id': 'req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.709233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] The token cache is being invalidate because project f2eef3c897bd434ca79f5ee4105de68b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:48.709517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.710173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.711801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 872/1744] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:48 2022] DELETE /identity/v3/projects/f2eef3c897bd434ca79f5ee4105de68b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:48.715808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:48.717278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:48.730446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa89abe-d53f-40cd-b549-825c1502a409 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:48.789240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:48.789900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:48.790230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:48.790563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] PATH_INFO: `/v3/users/61146deb25214c4281dd7dca18c73a23` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:48.791871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61146deb25214c4281dd7dca18c73a23)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:48.793788 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:48.841092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61146deb25214c4281dd7dca18c73a23'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.841743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61146deb25214c4281dd7dca18c73a23'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.848704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61146deb25214c4281dd7dca18c73a23'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.860807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61146deb25214c4281dd7dca18c73a23'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.861455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b604235d-4d6a-4f1d-8323-2c7ea36b6e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61146deb25214c4281dd7dca18c73a23'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:48.874298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 873/1745] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:48 2022] DELETE /identity/v3/users/61146deb25214c4281dd7dca18c73a23 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:49.160989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:49.162748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:49.185025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f32cd0-c872-4cc3-a3f9-d498c0e03eef admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:49.301402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:49.302022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.302298 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.302698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] PATH_INFO: `/v3/projects/44e0e1b069494f63af9b01a57936ee15` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.304894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44e0e1b069494f63af9b01a57936ee15)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:49.306909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:49.359384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44e0e1b069494f63af9b01a57936ee15', 'request_id': 'req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:49.365581 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44e0e1b069494f63af9b01a57936ee15', 'request_id': 'req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:49.374119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44e0e1b069494f63af9b01a57936ee15', 'request_id': 'req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:49.385629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] The token cache is being invalidate because project 44e0e1b069494f63af9b01a57936ee15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:49.385920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:49.386598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a3b92e65-bfcd-4e60-aec2-6e6eeaae5b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:49.388136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 873/1746] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:49 2022] DELETE /identity/v3/projects/44e0e1b069494f63af9b01a57936ee15 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:49.393511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ac1f3-c4b4-4af8-b213-275583c4bace None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.393951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ac1f3-c4b4-4af8-b213-275583c4bace None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.394059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ac1f3-c4b4-4af8-b213-275583c4bace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.415264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b42ac1f3-c4b4-4af8-b213-275583c4bace None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:49.416536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b42ac1f3-c4b4-4af8-b213-275583c4bace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:49.428269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:49.478976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 874/1747] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:49.486928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bd1d0-6408-4cb0-9b07-bcd0473eee95 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.487209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bd1d0-6408-4cb0-9b07-bcd0473eee95 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.487480 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bd1d0-6408-4cb0-9b07-bcd0473eee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.511850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-665bd1d0-6408-4cb0-9b07-bcd0473eee95 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:49.513057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-665bd1d0-6408-4cb0-9b07-bcd0473eee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:49.521081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 874/1748] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:49.525664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ac1f3-c4b4-4af8-b213-275583c4bace None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:49.535571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ac1f3-c4b4-4af8-b213-275583c4bace None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:49.541000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:49.541591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.541934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.542297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.542976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:49.543280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:49.544862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:49.556775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 875/1749] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:49.561358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bd1d0-6408-4cb0-9b07-bcd0473eee95 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:49.571744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bd1d0-6408-4cb0-9b07-bcd0473eee95 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:49.578868 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae6523-24aa-4954-82c4-a8dbde2d4675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:49.579600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6523-24aa-4954-82c4-a8dbde2d4675 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.579951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6523-24aa-4954-82c4-a8dbde2d4675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.580293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6523-24aa-4954-82c4-a8dbde2d4675 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.581067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae6523-24aa-4954-82c4-a8dbde2d4675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:49.583118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae6523-24aa-4954-82c4-a8dbde2d4675 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:49.603289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:49.620201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 875/1750] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:49 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:49.624565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:49.642919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ae054-72e7-44e5-8547-66ec3d0693dd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:49.703217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c08ba-4803-4d86-b214-047a11e595ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:49.703483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c08ba-4803-4d86-b214-047a11e595ea admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.703756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c08ba-4803-4d86-b214-047a11e595ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.704020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c08ba-4803-4d86-b214-047a11e595ea admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.704660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c08ba-4803-4d86-b214-047a11e595ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:49.706568 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c08ba-4803-4d86-b214-047a11e595ea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:49.732273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 876/1751] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:49 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:49.736624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae6523-24aa-4954-82c4-a8dbde2d4675 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:49.747187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae6523-24aa-4954-82c4-a8dbde2d4675 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:49.752370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01add6-6221-427c-bc96-25a6051cad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:49.752973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01add6-6221-427c-bc96-25a6051cad95 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.753251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01add6-6221-427c-bc96-25a6051cad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.753519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01add6-6221-427c-bc96-25a6051cad95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.754187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01add6-6221-427c-bc96-25a6051cad95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:49.755825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01add6-6221-427c-bc96-25a6051cad95 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:49.766629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 876/1752] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:25:49.771347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c08ba-4803-4d86-b214-047a11e595ea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:49.782421 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c08ba-4803-4d86-b214-047a11e595ea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:49.787065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aac763-9638-459c-8074-24dbc7f67b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:49.787658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac763-9638-459c-8074-24dbc7f67b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.787963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac763-9638-459c-8074-24dbc7f67b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.788235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac763-9638-459c-8074-24dbc7f67b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.789036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aac763-9638-459c-8074-24dbc7f67b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:49.790680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aac763-9638-459c-8074-24dbc7f67b3e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:49.802592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 877/1753] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:49.807198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01add6-6221-427c-bc96-25a6051cad95 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:49.817254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01add6-6221-427c-bc96-25a6051cad95 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:49.822634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:49.823316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.823588 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.823852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] PATH_INFO: `/v3/projects/c57c7346ceaf489ebb7959041930d242/users/3f4a2df151b94b4c8dc89c53fd618c24/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.839750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c57c7346ceaf489ebb7959041930d242, user_id=3f4a2df151b94b4c8dc89c53fd618c24, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:49.842419 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:49.856309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 877/1754] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:49 2022] PUT /identity/v3/projects/c57c7346ceaf489ebb7959041930d242/users/3f4a2df151b94b4c8dc89c53fd618c24/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:49.866724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a493eb-7bf5-4958-b5a7-5dc235ae569c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:49.867000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a493eb-7bf5-4958-b5a7-5dc235ae569c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:49.867271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a493eb-7bf5-4958-b5a7-5dc235ae569c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:49.896401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-52a493eb-7bf5-4958-b5a7-5dc235ae569c None None] MFA Rules not processed for user `3f4a2df151b94b4c8dc89c53fd618c24`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:49.897715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-52a493eb-7bf5-4958-b5a7-5dc235ae569c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:49.966422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 878/1755] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:49.978099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:49.979799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:50.034983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0132b14-7d7b-4e04-a86f-40801145867f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:50.039488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b788b-50ca-4950-8e16-8af6d31ff7aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:50.040080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b788b-50ca-4950-8e16-8af6d31ff7aa service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:50.040358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b788b-50ca-4950-8e16-8af6d31ff7aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:50.040620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b788b-50ca-4950-8e16-8af6d31ff7aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:50.048564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b788b-50ca-4950-8e16-8af6d31ff7aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:50.049966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b788b-50ca-4950-8e16-8af6d31ff7aa service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:50.059121 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 878/1756] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:50.086246 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a493eb-7bf5-4958-b5a7-5dc235ae569c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:50.100191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:50.151152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a493eb-7bf5-4958-b5a7-5dc235ae569c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:50.155743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:50.156362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:50.156635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:50.156901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] PATH_INFO: `/v3/users/3f4a2df151b94b4c8dc89c53fd618c24` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:50.158021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4a2df151b94b4c8dc89c53fd618c24)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:50.159951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:50.204939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4a2df151b94b4c8dc89c53fd618c24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.205604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4a2df151b94b4c8dc89c53fd618c24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.213537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4a2df151b94b4c8dc89c53fd618c24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.224992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4a2df151b94b4c8dc89c53fd618c24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.226190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4a2df151b94b4c8dc89c53fd618c24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.238146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 879/1757] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:50 2022] DELETE /identity/v3/users/3f4a2df151b94b4c8dc89c53fd618c24 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:50.243597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a1292-89c5-461d-8424-7eba7669d902 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:50.244077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a1292-89c5-461d-8424-7eba7669d902 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:50.244154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a1292-89c5-461d-8424-7eba7669d902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:50.265999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4a9a1292-89c5-461d-8424-7eba7669d902 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:50.267231 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a9a1292-89c5-461d-8424-7eba7669d902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:50.337046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 879/1758] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:50.339561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:50.340945 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:50.397077 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685eb790-1362-424e-bedd-3440140108ec admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:50.401233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349a37f-e83a-46dc-88ed-c3059a37faab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:50.401823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a37f-e83a-46dc-88ed-c3059a37faab service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:50.402097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a37f-e83a-46dc-88ed-c3059a37faab service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:50.402408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a37f-e83a-46dc-88ed-c3059a37faab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:50.411570 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349a37f-e83a-46dc-88ed-c3059a37faab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:50.413771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349a37f-e83a-46dc-88ed-c3059a37faab service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:50.424718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 880/1759] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:25:50.706450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a1292-89c5-461d-8424-7eba7669d902 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:50.707903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a9a1292-89c5-461d-8424-7eba7669d902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:50.721117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a1292-89c5-461d-8424-7eba7669d902 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:50.726274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:50.726938 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:50.727239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:50.727513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] PATH_INFO: `/v3/projects/c57c7346ceaf489ebb7959041930d242` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:50.728937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c57c7346ceaf489ebb7959041930d242)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:50.730924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:50.779967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c57c7346ceaf489ebb7959041930d242', 'request_id': 'req-9be388b6-36de-41a5-b7ae-e8119de97da6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.786135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c57c7346ceaf489ebb7959041930d242', 'request_id': 'req-9be388b6-36de-41a5-b7ae-e8119de97da6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.790445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:50.796664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c57c7346ceaf489ebb7959041930d242', 'request_id': 'req-9be388b6-36de-41a5-b7ae-e8119de97da6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.810454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] The token cache is being invalidate because project c57c7346ceaf489ebb7959041930d242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:50.810741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.811397 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9be388b6-36de-41a5-b7ae-e8119de97da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:50.812905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 880/1760] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:50 2022] DELETE /identity/v3/projects/c57c7346ceaf489ebb7959041930d242 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:50.817812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8c5fc-2e65-4593-90ae-add78defcd9f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:50.818030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8c5fc-2e65-4593-90ae-add78defcd9f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:50.818298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8c5fc-2e65-4593-90ae-add78defcd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:50.837639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fbf8c5fc-2e65-4593-90ae-add78defcd9f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:50.838868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fbf8c5fc-2e65-4593-90ae-add78defcd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:50.878853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:50.911840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 881/1761] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:50.921836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7cad7-1a48-4114-ba1a-cd571ca4f11a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:50.922354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7cad7-1a48-4114-ba1a-cd571ca4f11a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:50.922658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7cad7-1a48-4114-ba1a-cd571ca4f11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:50.943265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ceb7cad7-1a48-4114-ba1a-cd571ca4f11a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:50.944695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ceb7cad7-1a48-4114-ba1a-cd571ca4f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:50.952610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 881/1762] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:50.956585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8c5fc-2e65-4593-90ae-add78defcd9f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:50.966698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8c5fc-2e65-4593-90ae-add78defcd9f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:50.972886 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:50.973495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:50.973798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:50.974066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:50.974755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:25:50.975056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:50.977435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:50.990069 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 882/1763] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:25:50.994480 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7cad7-1a48-4114-ba1a-cd571ca4f11a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:51.003649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7cad7-1a48-4114-ba1a-cd571ca4f11a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:51.008393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f77940-346f-4cef-a002-d3e16c432250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:51.009009 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f77940-346f-4cef-a002-d3e16c432250 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:51.009266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f77940-346f-4cef-a002-d3e16c432250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:51.009533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f77940-346f-4cef-a002-d3e16c432250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:51.010249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f77940-346f-4cef-a002-d3e16c432250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:51.012194 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f77940-346f-4cef-a002-d3e16c432250 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:51.044970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 882/1764] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:25:50 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:51.046096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:51.056582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31ab48-81dc-42de-a5ce-65e827df046e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:51.117150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb615e-7201-4f58-aecb-f3963d075491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:51.117751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb615e-7201-4f58-aecb-f3963d075491 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:51.118028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb615e-7201-4f58-aecb-f3963d075491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:51.118299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb615e-7201-4f58-aecb-f3963d075491 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:51.118964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb615e-7201-4f58-aecb-f3963d075491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:51.120853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb615e-7201-4f58-aecb-f3963d075491 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:51.141748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 883/1765] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:25:51 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:25:51.146080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f77940-346f-4cef-a002-d3e16c432250 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:51.157053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f77940-346f-4cef-a002-d3e16c432250 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:51.161627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9847acf-1bd3-4e36-bb26-b24de728c1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:51.162222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9847acf-1bd3-4e36-bb26-b24de728c1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:51.162534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9847acf-1bd3-4e36-bb26-b24de728c1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:51.163678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9847acf-1bd3-4e36-bb26-b24de728c1e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:51.164376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9847acf-1bd3-4e36-bb26-b24de728c1e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:51.165978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9847acf-1bd3-4e36-bb26-b24de728c1e7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:51.176607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 883/1766] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:51.180838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb615e-7201-4f58-aecb-f3963d075491 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:51.190806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb615e-7201-4f58-aecb-f3963d075491 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:51.195579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b6c1-7ece-44e9-9af0-f70f20512456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:51.196163 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b6c1-7ece-44e9-9af0-f70f20512456 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:51.196438 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b6c1-7ece-44e9-9af0-f70f20512456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:51.196708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b6c1-7ece-44e9-9af0-f70f20512456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:51.197344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3b6c1-7ece-44e9-9af0-f70f20512456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:51.198970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3b6c1-7ece-44e9-9af0-f70f20512456 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:51.209296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 884/1767] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:25:51.213799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9847acf-1bd3-4e36-bb26-b24de728c1e7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:51.223640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9847acf-1bd3-4e36-bb26-b24de728c1e7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:51.229223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:51.229863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:51.230139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:51.230456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] PATH_INFO: `/v3/projects/c4cb0a2061864071b439139294498694/users/7c0a732376c64dd4bbeb8f7e16a93fed/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:51.245971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cb0a2061864071b439139294498694, user_id=7c0a732376c64dd4bbeb8f7e16a93fed, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:51.248853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:51.263116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 884/1768] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:25:51 2022] PUT /identity/v3/projects/c4cb0a2061864071b439139294498694/users/7c0a732376c64dd4bbeb8f7e16a93fed/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:51.274254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea21fa2-7fca-479a-81f7-9fea37e1cae7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:51.274581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea21fa2-7fca-479a-81f7-9fea37e1cae7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:51.274853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea21fa2-7fca-479a-81f7-9fea37e1cae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:51.302059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ea21fa2-7fca-479a-81f7-9fea37e1cae7 None None] MFA Rules not processed for user `7c0a732376c64dd4bbeb8f7e16a93fed`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:51.303495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ea21fa2-7fca-479a-81f7-9fea37e1cae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:51.373066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 885/1769] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:51.384536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:51.394924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:51.441939 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65be6ba-a048-475d-9413-dd482ac4848f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:51.446205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776740a7-10ee-499e-aa8d-9ea0d93871d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:51.446844 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776740a7-10ee-499e-aa8d-9ea0d93871d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:51.447119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776740a7-10ee-499e-aa8d-9ea0d93871d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:51.447386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776740a7-10ee-499e-aa8d-9ea0d93871d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:51.455486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776740a7-10ee-499e-aa8d-9ea0d93871d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:51.456897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776740a7-10ee-499e-aa8d-9ea0d93871d2 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:51.466156 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 885/1770] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:56.693730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea21fa2-7fca-479a-81f7-9fea37e1cae7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:56.695849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ea21fa2-7fca-479a-81f7-9fea37e1cae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:56.734846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:56.770090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea21fa2-7fca-479a-81f7-9fea37e1cae7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:56.774973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:56.775611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:56.776044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:56.776285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] PATH_INFO: `/v3/users/25356b2fbf32430ba2a0c6abee07b52f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:56.777448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25356b2fbf32430ba2a0c6abee07b52f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:56.779616 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:56.837183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25356b2fbf32430ba2a0c6abee07b52f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:56.837615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25356b2fbf32430ba2a0c6abee07b52f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:56.848693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25356b2fbf32430ba2a0c6abee07b52f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:56.870358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25356b2fbf32430ba2a0c6abee07b52f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:56.871311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25356b2fbf32430ba2a0c6abee07b52f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:56.887863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 886/1771] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:56 2022] DELETE /identity/v3/users/25356b2fbf32430ba2a0c6abee07b52f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:56.893888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa28f-59a2-4682-b8d2-64e89a076ee4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:56.895034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa28f-59a2-4682-b8d2-64e89a076ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:56.895578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa28f-59a2-4682-b8d2-64e89a076ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:56.947243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7deaa28f-59a2-4682-b8d2-64e89a076ee4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:56.948912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7deaa28f-59a2-4682-b8d2-64e89a076ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:57.033695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 886/1772] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:25:57.109016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:57.110843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:57.186789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8817-cde1-4395-96b2-78765f980cc4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:57.191132 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86f38c-d8f3-484e-8527-9d432726f278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:57.191917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86f38c-d8f3-484e-8527-9d432726f278 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:57.192293 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86f38c-d8f3-484e-8527-9d432726f278 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:57.192657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86f38c-d8f3-484e-8527-9d432726f278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:57.206902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86f38c-d8f3-484e-8527-9d432726f278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:57.210042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86f38c-d8f3-484e-8527-9d432726f278 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:57.232889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 887/1773] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:25:58.170569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaa28f-59a2-4682-b8d2-64e89a076ee4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:58.172107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7deaa28f-59a2-4682-b8d2-64e89a076ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:58.184632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaa28f-59a2-4682-b8d2-64e89a076ee4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:58.189298 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:58.189980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:58.190345 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:58.190855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] PATH_INFO: `/v3/projects/5106953452e747e6b7a44a0a4f19ebfa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:58.192035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5106953452e747e6b7a44a0a4f19ebfa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:58.194050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:58.219413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:58.266423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5106953452e747e6b7a44a0a4f19ebfa', 'request_id': 'req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.275116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5106953452e747e6b7a44a0a4f19ebfa', 'request_id': 'req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.291604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5106953452e747e6b7a44a0a4f19ebfa', 'request_id': 'req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.305625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] The token cache is being invalidate because project 5106953452e747e6b7a44a0a4f19ebfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:58.305912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.306798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.308614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 887/1774] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:58 2022] DELETE /identity/v3/projects/5106953452e747e6b7a44a0a4f19ebfa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:58.312650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86f38c-d8f3-484e-8527-9d432726f278 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:58.323409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86f38c-d8f3-484e-8527-9d432726f278 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:58.384368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:58.396905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:58.399384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:58.400497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:58.401204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] PATH_INFO: `/v3/users/5431496280fb4cd0b1072e860235e1a8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:58.402655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5431496280fb4cd0b1072e860235e1a8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:58.405346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:58.517001 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5431496280fb4cd0b1072e860235e1a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.517655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5431496280fb4cd0b1072e860235e1a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.526637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5431496280fb4cd0b1072e860235e1a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.540526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5431496280fb4cd0b1072e860235e1a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.541303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5431496280fb4cd0b1072e860235e1a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:58.555314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 888/1775] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:58 2022] DELETE /identity/v3/users/5431496280fb4cd0b1072e860235e1a8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:58.889786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:58.891281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:58.924044 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60d237-2d71-4beb-9bcb-fb1e72a5c342 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:59.024061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:59.024723 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:59.025000 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:59.025269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] PATH_INFO: `/v3/projects/6e86ea67982643879aa4bb140a416aa7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:59.027146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e86ea67982643879aa4bb140a416aa7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:59.029207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:59.102344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e86ea67982643879aa4bb140a416aa7', 'request_id': 'req-fff650a8-c4f9-4cd9-9877-7d0abbac434d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.109329 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e86ea67982643879aa4bb140a416aa7', 'request_id': 'req-fff650a8-c4f9-4cd9-9877-7d0abbac434d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.119903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e86ea67982643879aa4bb140a416aa7', 'request_id': 'req-fff650a8-c4f9-4cd9-9877-7d0abbac434d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.132123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] The token cache is being invalidate because project 6e86ea67982643879aa4bb140a416aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:59.132366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.133009 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.134888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 888/1776] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:58 2022] DELETE /identity/v3/projects/6e86ea67982643879aa4bb140a416aa7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:59.138947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:59.141048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:59.155869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f98a5-951f-4e33-abf8-40d6e85d37f2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:59.228907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:59.229546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:59.229821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:59.230111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] PATH_INFO: `/v3/users/72a0d1a94fa1404a90bcf6b6cf613a94` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:59.231366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a0d1a94fa1404a90bcf6b6cf613a94)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:59.233277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:59.291888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a0d1a94fa1404a90bcf6b6cf613a94'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.293021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a0d1a94fa1404a90bcf6b6cf613a94'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.311087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a0d1a94fa1404a90bcf6b6cf613a94'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.341216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a0d1a94fa1404a90bcf6b6cf613a94'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.342458 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78b2b1b8-f59c-4795-8504-354498ba55b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a0d1a94fa1404a90bcf6b6cf613a94'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.345104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:59.366449 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 889/1777] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:25:59 2022] DELETE /identity/v3/users/72a0d1a94fa1404a90bcf6b6cf613a94 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:25:59.673373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:25:59.674982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:59.692203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff650a8-c4f9-4cd9-9877-7d0abbac434d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:25:59.755821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:25:59.794592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:25:59.795189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:59.795486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:59.795781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] PATH_INFO: `/v3/projects/518cd00e397242a2a04d8fdda6249751` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:59.796982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518cd00e397242a2a04d8fdda6249751)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:25:59.799813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:25:59.855777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518cd00e397242a2a04d8fdda6249751', 'request_id': 'req-911798d0-4d7f-4f0a-82e0-965467b4e121'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.862198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518cd00e397242a2a04d8fdda6249751', 'request_id': 'req-911798d0-4d7f-4f0a-82e0-965467b4e121'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.872592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518cd00e397242a2a04d8fdda6249751', 'request_id': 'req-911798d0-4d7f-4f0a-82e0-965467b4e121'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.884839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] The token cache is being invalidate because project 518cd00e397242a2a04d8fdda6249751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:25:59.885118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.885945 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-911798d0-4d7f-4f0a-82e0-965467b4e121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:25:59.887590 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 889/1778] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:25:59 2022] DELETE /identity/v3/projects/518cd00e397242a2a04d8fdda6249751 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:25:59.894615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a6485-7aa4-4531-af73-1b7ece008c56 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:59.894927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a6485-7aa4-4531-af73-1b7ece008c56 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:59.895194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a6485-7aa4-4531-af73-1b7ece008c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:25:59.917110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a99a6485-7aa4-4531-af73-1b7ece008c56 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:25:59.918229 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a99a6485-7aa4-4531-af73-1b7ece008c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:25:59.980668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 890/1779] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:25:59.988886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0feff-db38-4998-a8e7-77797a306190 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:25:59.989076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0feff-db38-4998-a8e7-77797a306190 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:25:59.989359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0feff-db38-4998-a8e7-77797a306190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.013038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-31f0feff-db38-4998-a8e7-77797a306190 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:00.014611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-31f0feff-db38-4998-a8e7-77797a306190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:00.025580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 890/1780] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:00.029871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a6485-7aa4-4531-af73-1b7ece008c56 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:00.041586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a6485-7aa4-4531-af73-1b7ece008c56 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:00.046222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:00.046852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:00.047126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:00.047390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.048034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:00.048336 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:00.050048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:00.062747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 891/1781] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:26:00.067222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0feff-db38-4998-a8e7-77797a306190 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:00.092790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0feff-db38-4998-a8e7-77797a306190 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:00.099422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d117a1-8287-41e3-8a31-085c5264c769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:00.099894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d117a1-8287-41e3-8a31-085c5264c769 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:00.100177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d117a1-8287-41e3-8a31-085c5264c769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:00.100452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d117a1-8287-41e3-8a31-085c5264c769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.101112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d117a1-8287-41e3-8a31-085c5264c769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:00.104196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d117a1-8287-41e3-8a31-085c5264c769 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:00.147043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 891/1782] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:00 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:00.151680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:00.161863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a058748-90a1-4a1d-bbb3-5a3fbb513600 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:00.236418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e5d06-60ba-421c-9f65-e896a0849daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:00.237132 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e5d06-60ba-421c-9f65-e896a0849daf admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:00.237484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e5d06-60ba-421c-9f65-e896a0849daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:00.237829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e5d06-60ba-421c-9f65-e896a0849daf admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.238736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e5d06-60ba-421c-9f65-e896a0849daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:00.240537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e5d06-60ba-421c-9f65-e896a0849daf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:00.264290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 892/1783] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:00 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:26:00.268410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d117a1-8287-41e3-8a31-085c5264c769 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:00.279671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d117a1-8287-41e3-8a31-085c5264c769 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:00.284811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3d022-74d7-4577-b661-7d7781e1b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:00.285421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d022-74d7-4577-b661-7d7781e1b28e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:00.285702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d022-74d7-4577-b661-7d7781e1b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:00.285975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d022-74d7-4577-b661-7d7781e1b28e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.286650 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3d022-74d7-4577-b661-7d7781e1b28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:00.288270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3d022-74d7-4577-b661-7d7781e1b28e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:00.299161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 892/1784] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:00.303107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e5d06-60ba-421c-9f65-e896a0849daf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:00.313306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e5d06-60ba-421c-9f65-e896a0849daf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:00.317905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e85ce-0970-438d-b2bb-1d6f40512c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:00.318525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e85ce-0970-438d-b2bb-1d6f40512c0a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:00.318831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e85ce-0970-438d-b2bb-1d6f40512c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:00.319100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e85ce-0970-438d-b2bb-1d6f40512c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.319739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e85ce-0970-438d-b2bb-1d6f40512c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:00.322019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e85ce-0970-438d-b2bb-1d6f40512c0a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:00.332400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 893/1785] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:00.336368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3d022-74d7-4577-b661-7d7781e1b28e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:00.348441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3d022-74d7-4577-b661-7d7781e1b28e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:00.354606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:00.355330 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:00.355674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:00.356012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] PATH_INFO: `/v3/projects/0e87741d68c74ae29ecea930893e3449/users/a5b7190649694b9882ab34c0f3114a3e/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.371127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e87741d68c74ae29ecea930893e3449, user_id=a5b7190649694b9882ab34c0f3114a3e, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:00.373816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:00.386977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 893/1786] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:00 2022] PUT /identity/v3/projects/0e87741d68c74ae29ecea930893e3449/users/a5b7190649694b9882ab34c0f3114a3e/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:00.391963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892097-293f-42cd-9d2c-5f26ed96ab13 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:00.392477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892097-293f-42cd-9d2c-5f26ed96ab13 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:00.392561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892097-293f-42cd-9d2c-5f26ed96ab13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.411747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2c892097-293f-42cd-9d2c-5f26ed96ab13 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:00.412933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2c892097-293f-42cd-9d2c-5f26ed96ab13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:00.462374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:00.472448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 894/1787] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:00.482308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:00.483997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:00.541637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53703baf-f1ab-4249-bb9b-c7f96ba2493b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:00.545292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0216831-c2b6-45a1-a737-499a96acfd3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:00.545887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216831-c2b6-45a1-a737-499a96acfd3c service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:00.546161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216831-c2b6-45a1-a737-499a96acfd3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:00.546460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216831-c2b6-45a1-a737-499a96acfd3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:00.555755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0216831-c2b6-45a1-a737-499a96acfd3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:00.557166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0216831-c2b6-45a1-a737-499a96acfd3c service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:00.566303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 894/1788] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:05.112216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92660a77-983c-4964-bf8d-953acd83a8d0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:05.112822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92660a77-983c-4964-bf8d-953acd83a8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:05.112822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92660a77-983c-4964-bf8d-953acd83a8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:05.146669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-92660a77-983c-4964-bf8d-953acd83a8d0 None None] MFA Rules not processed for user `a5b7190649694b9882ab34c0f3114a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:05.147915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-92660a77-983c-4964-bf8d-953acd83a8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:05.228712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 895/1789] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:05.233345 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0216831-c2b6-45a1-a737-499a96acfd3c service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:05.243637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0216831-c2b6-45a1-a737-499a96acfd3c service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:05.248316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5379c-e9b6-4be4-be07-a4237b0d5e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:05.248927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5379c-e9b6-4be4-be07-a4237b0d5e85 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:05.249203 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5379c-e9b6-4be4-be07-a4237b0d5e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:05.249475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5379c-e9b6-4be4-be07-a4237b0d5e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:05.250132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5379c-e9b6-4be4-be07-a4237b0d5e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:05.252107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5379c-e9b6-4be4-be07-a4237b0d5e85 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:05.256843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:05.293205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 895/1790] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:05 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:05.294561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92660a77-983c-4964-bf8d-953acd83a8d0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:05.305747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92660a77-983c-4964-bf8d-953acd83a8d0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:05.371845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73915b3-07bf-4acf-8d1d-34589d3719da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:05.372451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73915b3-07bf-4acf-8d1d-34589d3719da admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:05.372721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73915b3-07bf-4acf-8d1d-34589d3719da admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:05.372983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73915b3-07bf-4acf-8d1d-34589d3719da admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:05.373613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73915b3-07bf-4acf-8d1d-34589d3719da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:05.375549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73915b3-07bf-4acf-8d1d-34589d3719da admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:05.397933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 896/1791] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:05 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:05.402473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5379c-e9b6-4be4-be07-a4237b0d5e85 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:05.412823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5379c-e9b6-4be4-be07-a4237b0d5e85 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:05.418136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d590-0781-40e2-98c3-b7ab50d192e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:05.418774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d590-0781-40e2-98c3-b7ab50d192e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:05.419049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d590-0781-40e2-98c3-b7ab50d192e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:05.419320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d590-0781-40e2-98c3-b7ab50d192e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:05.419965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d590-0781-40e2-98c3-b7ab50d192e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:05.421559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d590-0781-40e2-98c3-b7ab50d192e7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:05.433007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 896/1792] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:05.436996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73915b3-07bf-4acf-8d1d-34589d3719da admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:05.447931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73915b3-07bf-4acf-8d1d-34589d3719da admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:05.453107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7f4d8-7657-40ed-bdd2-6956fc8900b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:05.453537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f4d8-7657-40ed-bdd2-6956fc8900b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:05.453818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f4d8-7657-40ed-bdd2-6956fc8900b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:05.454094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f4d8-7657-40ed-bdd2-6956fc8900b4 admin admin] PATH_INFO: `/v3/projects/4d06020a20de48389f76209233fea396/users/839c4fdf3b1a4506b7d0ca13629e9242/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:05.471048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7f4d8-7657-40ed-bdd2-6956fc8900b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d06020a20de48389f76209233fea396, user_id=839c4fdf3b1a4506b7d0ca13629e9242, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:05.474243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7f4d8-7657-40ed-bdd2-6956fc8900b4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:05.496939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 897/1793] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:05 2022] PUT /identity/v3/projects/4d06020a20de48389f76209233fea396/users/839c4fdf3b1a4506b7d0ca13629e9242/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:05.500941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d590-0781-40e2-98c3-b7ab50d192e7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:05.511530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d590-0781-40e2-98c3-b7ab50d192e7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:05.566383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb292ab-1f0e-4af8-b72d-c1bb565ce49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:05.566879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb292ab-1f0e-4af8-b72d-c1bb565ce49b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:05.567158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb292ab-1f0e-4af8-b72d-c1bb565ce49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:05.567427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb292ab-1f0e-4af8-b72d-c1bb565ce49b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:05.568059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb292ab-1f0e-4af8-b72d-c1bb565ce49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:05.569649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb292ab-1f0e-4af8-b72d-c1bb565ce49b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:05.579751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 897/1794] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:05.583937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7f4d8-7657-40ed-bdd2-6956fc8900b4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:05.595426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7f4d8-7657-40ed-bdd2-6956fc8900b4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:05.600045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:05.600683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:05.600953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:05.601221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] PATH_INFO: `/v3/projects/4d06020a20de48389f76209233fea396/users/839c4fdf3b1a4506b7d0ca13629e9242/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:05.603929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d06020a20de48389f76209233fea396, user_id=839c4fdf3b1a4506b7d0ca13629e9242, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:05.606558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:05.616609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 898/1795] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:05 2022] PUT /identity/v3/projects/4d06020a20de48389f76209233fea396/users/839c4fdf3b1a4506b7d0ca13629e9242/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:11.376080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64f16a-25cd-4955-83f0-486c8c78d893 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:11.376610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64f16a-25cd-4955-83f0-486c8c78d893 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:11.376755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64f16a-25cd-4955-83f0-486c8c78d893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:11.406812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3a64f16a-25cd-4955-83f0-486c8c78d893 None None] MFA Rules not processed for user `839c4fdf3b1a4506b7d0ca13629e9242`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:11.408090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3a64f16a-25cd-4955-83f0-486c8c78d893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:11.486524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 898/1796] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:11.500808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:11.502646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:11.549510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:11.584142 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed268bb3-6804-4eb4-bf6a-f9e4434ab63c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:11.590994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1106d8-9ee2-46f2-a867-432c1bd4724d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:11.591611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1106d8-9ee2-46f2-a867-432c1bd4724d service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:11.591906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1106d8-9ee2-46f2-a867-432c1bd4724d service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:11.592172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1106d8-9ee2-46f2-a867-432c1bd4724d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:11.656111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1106d8-9ee2-46f2-a867-432c1bd4724d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:11.657703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1106d8-9ee2-46f2-a867-432c1bd4724d service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:11.667090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 899/1797] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:21.483368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:21.662762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=38 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:25.644637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64f16a-25cd-4955-83f0-486c8c78d893 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:25.646279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3a64f16a-25cd-4955-83f0-486c8c78d893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:25.733531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64f16a-25cd-4955-83f0-486c8c78d893 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:25.738812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:25.740373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:25.740727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:25.741012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] PATH_INFO: `/v3/users/516cb672c22d48328c2fe5f1ffd3d96d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:25.742974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=516cb672c22d48328c2fe5f1ffd3d96d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:25.747672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:25.795655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516cb672c22d48328c2fe5f1ffd3d96d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:25.796306 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516cb672c22d48328c2fe5f1ffd3d96d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:25.809411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516cb672c22d48328c2fe5f1ffd3d96d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:25.816370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516cb672c22d48328c2fe5f1ffd3d96d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:25.816978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516cb672c22d48328c2fe5f1ffd3d96d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:25.829633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 899/1798] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:25 2022] DELETE /identity/v3/users/516cb672c22d48328c2fe5f1ffd3d96d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:25.835036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed151ed-82c9-443c-bd42-e0e15638ec59 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:25.835686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed151ed-82c9-443c-bd42-e0e15638ec59 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:25.835686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed151ed-82c9-443c-bd42-e0e15638ec59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:25.858446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7ed151ed-82c9-443c-bd42-e0e15638ec59 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:25.859979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7ed151ed-82c9-443c-bd42-e0e15638ec59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:25.921810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 900/1799] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:25.934451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:25.935907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:26.000322 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127466c-4d33-4e39-a118-cd81795b2a67 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:26.005290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa877e9-1c7f-4518-a775-273dc48050de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:26.006040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa877e9-1c7f-4518-a775-273dc48050de service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:26.006522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa877e9-1c7f-4518-a775-273dc48050de service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:26.006941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa877e9-1c7f-4518-a775-273dc48050de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:26.019259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa877e9-1c7f-4518-a775-273dc48050de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:26.020749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa877e9-1c7f-4518-a775-273dc48050de service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:26.031245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 900/1800] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:26.360231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed151ed-82c9-443c-bd42-e0e15638ec59 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:26.361813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7ed151ed-82c9-443c-bd42-e0e15638ec59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:26.379161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed151ed-82c9-443c-bd42-e0e15638ec59 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:26.395165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:26.395910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:26.396260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:26.396590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] PATH_INFO: `/v3/projects/efc04e251d344915a5460b91903dc811` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:26.398439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc04e251d344915a5460b91903dc811)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:26.401152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:26.474355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc04e251d344915a5460b91903dc811', 'request_id': 'req-a93c808d-08fd-4142-b0d9-dae1658fc056'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.474355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc04e251d344915a5460b91903dc811', 'request_id': 'req-a93c808d-08fd-4142-b0d9-dae1658fc056'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.479490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc04e251d344915a5460b91903dc811', 'request_id': 'req-a93c808d-08fd-4142-b0d9-dae1658fc056'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.494579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] The token cache is being invalidate because project efc04e251d344915a5460b91903dc811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:26.494871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.495496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.497000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 901/1801] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:26 2022] DELETE /identity/v3/projects/efc04e251d344915a5460b91903dc811 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:26.501409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa877e9-1c7f-4518-a775-273dc48050de service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:26.513151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa877e9-1c7f-4518-a775-273dc48050de service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:26.533668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:26.584154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:26.584933 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:26.585220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:26.586073 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] PATH_INFO: `/v3/users/848f048a6e11428b906259d371838864` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:26.587566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] RBAC: Authorizing `identity:delete_user(user_id=848f048a6e11428b906259d371838864)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:26.589509 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:26.637787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848f048a6e11428b906259d371838864'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.638484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848f048a6e11428b906259d371838864'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.655321 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848f048a6e11428b906259d371838864'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.663083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848f048a6e11428b906259d371838864'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.663809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0e3aa6a2-b77f-4657-a625-d7bacb4b636b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848f048a6e11428b906259d371838864'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:26.676983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 901/1802] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:26 2022] DELETE /identity/v3/users/848f048a6e11428b906259d371838864 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:27.192613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:27.194696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:27.211076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c808d-08fd-4142-b0d9-dae1658fc056 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:27.283699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:27.284317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.284594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.284862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] PATH_INFO: `/v3/projects/387697963916491b869f36b7b0ada4cf` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.285975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387697963916491b869f36b7b0ada4cf)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:27.287915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:27.318944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:27.337914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387697963916491b869f36b7b0ada4cf', 'request_id': 'req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:27.344371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387697963916491b869f36b7b0ada4cf', 'request_id': 'req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:27.349890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387697963916491b869f36b7b0ada4cf', 'request_id': 'req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:27.362641 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] The token cache is being invalidate because project 387697963916491b869f36b7b0ada4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:27.362925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:27.363536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cee5af09-8c9e-4ff6-9f71-8a149539b7d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:27.365046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 902/1803] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:27 2022] DELETE /identity/v3/projects/387697963916491b869f36b7b0ada4cf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:27.370527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffede1ad-f62d-4e60-97b0-a5ff7c16c6f2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.370814 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffede1ad-f62d-4e60-97b0-a5ff7c16c6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.371085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffede1ad-f62d-4e60-97b0-a5ff7c16c6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.390405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ffede1ad-f62d-4e60-97b0-a5ff7c16c6f2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:27.391598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ffede1ad-f62d-4e60-97b0-a5ff7c16c6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:27.454741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 902/1804] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:27.460953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8de1c-8200-4f01-8678-abd525993c20 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.461269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8de1c-8200-4f01-8678-abd525993c20 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.461548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8de1c-8200-4f01-8678-abd525993c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.482308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fee8de1c-8200-4f01-8678-abd525993c20 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:27.483528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fee8de1c-8200-4f01-8678-abd525993c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:27.491505 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 903/1805] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:27.496328 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffede1ad-f62d-4e60-97b0-a5ff7c16c6f2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:27.505974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffede1ad-f62d-4e60-97b0-a5ff7c16c6f2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:27.511988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:27.512612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.512885 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.513157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.513805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:27.514102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:27.515776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:27.528971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 903/1806] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:26:27.532966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8de1c-8200-4f01-8678-abd525993c20 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:27.542768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8de1c-8200-4f01-8678-abd525993c20 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:27.547410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2fb81-fce0-4491-8b58-66e9f7b79017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:27.548008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fb81-fce0-4491-8b58-66e9f7b79017 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.548281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fb81-fce0-4491-8b58-66e9f7b79017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.548544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fb81-fce0-4491-8b58-66e9f7b79017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.549180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2fb81-fce0-4491-8b58-66e9f7b79017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:27.551112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2fb81-fce0-4491-8b58-66e9f7b79017 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:27.586996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 904/1807] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:27 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:27.587976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:27.597956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47fabd-2b8e-4479-9b13-e761ae198f58 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:27.629314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:27.663494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e0c7d-0309-43d6-a068-a010fb99b1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:27.664642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e0c7d-0309-43d6-a068-a010fb99b1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.664861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e0c7d-0309-43d6-a068-a010fb99b1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.665123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e0c7d-0309-43d6-a068-a010fb99b1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.665784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e0c7d-0309-43d6-a068-a010fb99b1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:27.667723 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e0c7d-0309-43d6-a068-a010fb99b1e5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:27.691045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 904/1808] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:27 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:26:27.692801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2fb81-fce0-4491-8b58-66e9f7b79017 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:27.703182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2fb81-fce0-4491-8b58-66e9f7b79017 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:27.707933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe75115-f1c5-4d25-9860-086f8ccd3c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:27.708542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75115-f1c5-4d25-9860-086f8ccd3c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.708884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75115-f1c5-4d25-9860-086f8ccd3c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.709183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75115-f1c5-4d25-9860-086f8ccd3c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.709838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe75115-f1c5-4d25-9860-086f8ccd3c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:27.712180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe75115-f1c5-4d25-9860-086f8ccd3c31 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:27.722923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 905/1809] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:27.727463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e0c7d-0309-43d6-a068-a010fb99b1e5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:27.737557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e0c7d-0309-43d6-a068-a010fb99b1e5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:27.742385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e78118-c9f4-4d78-bb67-cce0f31c31b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:27.743000 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78118-c9f4-4d78-bb67-cce0f31c31b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.743277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78118-c9f4-4d78-bb67-cce0f31c31b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.743576 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78118-c9f4-4d78-bb67-cce0f31c31b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.744437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e78118-c9f4-4d78-bb67-cce0f31c31b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:27.746467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e78118-c9f4-4d78-bb67-cce0f31c31b3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:27.761722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 905/1810] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:27.762952 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe75115-f1c5-4d25-9860-086f8ccd3c31 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:27.773737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe75115-f1c5-4d25-9860-086f8ccd3c31 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:27.778559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d75279-24ec-4b59-8cd9-b5f36a7b576e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:27.779237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d75279-24ec-4b59-8cd9-b5f36a7b576e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.779516 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d75279-24ec-4b59-8cd9-b5f36a7b576e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.779962 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d75279-24ec-4b59-8cd9-b5f36a7b576e admin admin] PATH_INFO: `/v3/projects/d1ba4294f94d44b5bf1fccce8116253d/users/8d907205419d4b27985cab63160b5d9b/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.796918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d75279-24ec-4b59-8cd9-b5f36a7b576e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ba4294f94d44b5bf1fccce8116253d, user_id=8d907205419d4b27985cab63160b5d9b, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:27.799702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d75279-24ec-4b59-8cd9-b5f36a7b576e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:27.832996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 906/1811] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:27 2022] PUT /identity/v3/projects/d1ba4294f94d44b5bf1fccce8116253d/users/8d907205419d4b27985cab63160b5d9b/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:27.836433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18b4c6-8993-4464-bb24-329d5f1119f4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:27.836767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18b4c6-8993-4464-bb24-329d5f1119f4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:27.837094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18b4c6-8993-4464-bb24-329d5f1119f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:27.877288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7c18b4c6-8993-4464-bb24-329d5f1119f4 None None] MFA Rules not processed for user `8d907205419d4b27985cab63160b5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:27.878671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7c18b4c6-8993-4464-bb24-329d5f1119f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:27.959304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 906/1812] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:26:27.964585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d75279-24ec-4b59-8cd9-b5f36a7b576e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:28.038102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d75279-24ec-4b59-8cd9-b5f36a7b576e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:28.043585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa4c0b-07f4-4696-b946-65d5442fa135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:28.044291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa4c0b-07f4-4696-b946-65d5442fa135 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:28.044707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa4c0b-07f4-4696-b946-65d5442fa135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:28.045031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa4c0b-07f4-4696-b946-65d5442fa135 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:28.047063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa4c0b-07f4-4696-b946-65d5442fa135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:28.049166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa4c0b-07f4-4696-b946-65d5442fa135 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:28.080095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 907/1813] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:27 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:28.083974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18b4c6-8993-4464-bb24-329d5f1119f4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:28.097057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18b4c6-8993-4464-bb24-329d5f1119f4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:28.170921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072f963-d21e-4d72-8716-e468a194df66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:28.171541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072f963-d21e-4d72-8716-e468a194df66 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:28.171878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072f963-d21e-4d72-8716-e468a194df66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:28.172124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072f963-d21e-4d72-8716-e468a194df66 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:28.172983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072f963-d21e-4d72-8716-e468a194df66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:28.175081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072f963-d21e-4d72-8716-e468a194df66 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:28.204684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 907/1814] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:28 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:26:28.208481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa4c0b-07f4-4696-b946-65d5442fa135 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:28.220113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa4c0b-07f4-4696-b946-65d5442fa135 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:28.224844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4774bb6-cff5-4bd8-8f87-a85ce2f638ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:28.225507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774bb6-cff5-4bd8-8f87-a85ce2f638ed admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:28.225782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774bb6-cff5-4bd8-8f87-a85ce2f638ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:28.226160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774bb6-cff5-4bd8-8f87-a85ce2f638ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:28.226825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4774bb6-cff5-4bd8-8f87-a85ce2f638ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:28.228715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4774bb6-cff5-4bd8-8f87-a85ce2f638ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:28.239933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 908/1815] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:28.247728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072f963-d21e-4d72-8716-e468a194df66 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:28.258826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072f963-d21e-4d72-8716-e468a194df66 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:28.263390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e765c-f199-4320-905c-32fbcc5d50f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:28.264081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e765c-f199-4320-905c-32fbcc5d50f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:28.264357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e765c-f199-4320-905c-32fbcc5d50f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:28.264628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e765c-f199-4320-905c-32fbcc5d50f0 admin admin] PATH_INFO: `/v3/projects/41726e6bc1854b9da0661168174e8776/users/7324a8324a694a9bb0c4aac1d415bb81/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:28.283891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e765c-f199-4320-905c-32fbcc5d50f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41726e6bc1854b9da0661168174e8776, user_id=7324a8324a694a9bb0c4aac1d415bb81, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:28.287111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e765c-f199-4320-905c-32fbcc5d50f0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:28.303790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 908/1816] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:28 2022] PUT /identity/v3/projects/41726e6bc1854b9da0661168174e8776/users/7324a8324a694a9bb0c4aac1d415bb81/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:28.308028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4774bb6-cff5-4bd8-8f87-a85ce2f638ed admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:28.321023 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4774bb6-cff5-4bd8-8f87-a85ce2f638ed admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:28.356515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:28.399617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457f134-4370-4211-a17c-ec3daa8a8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:28.400243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457f134-4370-4211-a17c-ec3daa8a8590 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:28.400518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457f134-4370-4211-a17c-ec3daa8a8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:28.400784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457f134-4370-4211-a17c-ec3daa8a8590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:28.401454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457f134-4370-4211-a17c-ec3daa8a8590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:28.403073 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457f134-4370-4211-a17c-ec3daa8a8590 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:28.413111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 909/1817] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:28.417219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e765c-f199-4320-905c-32fbcc5d50f0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:28.422901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:28.428808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e765c-f199-4320-905c-32fbcc5d50f0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:28.435044 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:28.435856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:28.435967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:28.436475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] PATH_INFO: `/v3/projects/41726e6bc1854b9da0661168174e8776/users/7324a8324a694a9bb0c4aac1d415bb81/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:28.438210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] RBAC: Authorizing `identity:create_grant(project_id=41726e6bc1854b9da0661168174e8776, user_id=7324a8324a694a9bb0c4aac1d415bb81, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:28.441246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:28.453592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 909/1818] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:28 2022] PUT /identity/v3/projects/41726e6bc1854b9da0661168174e8776/users/7324a8324a694a9bb0c4aac1d415bb81/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:28.464002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469774-ea1b-493a-ad46-bdac410ae0a8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:28.464282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469774-ea1b-493a-ad46-bdac410ae0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:28.464555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469774-ea1b-493a-ad46-bdac410ae0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:28.495040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3a469774-ea1b-493a-ad46-bdac410ae0a8 None None] MFA Rules not processed for user `7324a8324a694a9bb0c4aac1d415bb81`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:28.496563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3a469774-ea1b-493a-ad46-bdac410ae0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:28.571967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 910/1819] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:28.582704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:28.584497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:28.647002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecd3d8-9e41-4822-87d8-8f3ffb6a803c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:28.651098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a1e10-940a-4dba-92fc-5ed5a9adeb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:28.651733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1e10-940a-4dba-92fc-5ed5a9adeb1f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:28.652008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1e10-940a-4dba-92fc-5ed5a9adeb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:28.652276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1e10-940a-4dba-92fc-5ed5a9adeb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:28.661246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a1e10-940a-4dba-92fc-5ed5a9adeb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:28.662883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a1e10-940a-4dba-92fc-5ed5a9adeb1f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:28.673881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 910/1820] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:29.307233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a469774-ea1b-493a-ad46-bdac410ae0a8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:29.370933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a469774-ea1b-493a-ad46-bdac410ae0a8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:29.375480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:29.376108 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:29.376380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:29.376647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] PATH_INFO: `/v3/users/8d907205419d4b27985cab63160b5d9b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:29.377764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d907205419d4b27985cab63160b5d9b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:29.379655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:29.427004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d907205419d4b27985cab63160b5d9b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:29.427645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d907205419d4b27985cab63160b5d9b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:29.429819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:29.436309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d907205419d4b27985cab63160b5d9b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:29.449318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d907205419d4b27985cab63160b5d9b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:29.449904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d907205419d4b27985cab63160b5d9b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:29.461928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 911/1821] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:29 2022] DELETE /identity/v3/users/8d907205419d4b27985cab63160b5d9b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:29.466614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544b7e-07db-4028-8ac0-2efc5d261d84 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:29.466884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544b7e-07db-4028-8ac0-2efc5d261d84 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:29.467152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544b7e-07db-4028-8ac0-2efc5d261d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:29.487090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fb544b7e-07db-4028-8ac0-2efc5d261d84 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:29.488261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb544b7e-07db-4028-8ac0-2efc5d261d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:29.552236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 911/1822] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:29.562969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:29.564339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:29.623339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac8b72-5ad0-4c42-b2b0-76318ef614c5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:29.627013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213dd94-79b5-4603-a359-b0f860f07452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:29.627620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dd94-79b5-4603-a359-b0f860f07452 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:29.627895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dd94-79b5-4603-a359-b0f860f07452 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:29.628163 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dd94-79b5-4603-a359-b0f860f07452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:29.637480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213dd94-79b5-4603-a359-b0f860f07452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:29.638939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213dd94-79b5-4603-a359-b0f860f07452 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:29.648327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 912/1823] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:29.952324 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544b7e-07db-4028-8ac0-2efc5d261d84 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:29.953957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb544b7e-07db-4028-8ac0-2efc5d261d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:29.966741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544b7e-07db-4028-8ac0-2efc5d261d84 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:29.980283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:29.981168 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:29.981601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:29.981921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] PATH_INFO: `/v3/projects/d1ba4294f94d44b5bf1fccce8116253d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:29.983011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ba4294f94d44b5bf1fccce8116253d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:29.985067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:30.042673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ba4294f94d44b5bf1fccce8116253d', 'request_id': 'req-589a4367-ae09-40c6-aa6d-c378b4fd001b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.048954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ba4294f94d44b5bf1fccce8116253d', 'request_id': 'req-589a4367-ae09-40c6-aa6d-c378b4fd001b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.056877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ba4294f94d44b5bf1fccce8116253d', 'request_id': 'req-589a4367-ae09-40c6-aa6d-c378b4fd001b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.068901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] The token cache is being invalidate because project d1ba4294f94d44b5bf1fccce8116253d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:30.068901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.069649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.071816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 912/1824] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:29 2022] DELETE /identity/v3/projects/d1ba4294f94d44b5bf1fccce8116253d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:30.075763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213dd94-79b5-4603-a359-b0f860f07452 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:30.088214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213dd94-79b5-4603-a359-b0f860f07452 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:30.148525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:30.149165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:30.149473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:30.149744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] PATH_INFO: `/v3/users/7324a8324a694a9bb0c4aac1d415bb81` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:30.151052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7324a8324a694a9bb0c4aac1d415bb81)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:30.152976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:30.206466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7324a8324a694a9bb0c4aac1d415bb81'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.207372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7324a8324a694a9bb0c4aac1d415bb81'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.216024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7324a8324a694a9bb0c4aac1d415bb81'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.231441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:30.233752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7324a8324a694a9bb0c4aac1d415bb81'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.235076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a48a635-ebdc-4054-bf21-fb490d056394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7324a8324a694a9bb0c4aac1d415bb81'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.253094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 913/1825] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:30 2022] DELETE /identity/v3/users/7324a8324a694a9bb0c4aac1d415bb81 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:30.563389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:30.564965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:30.582474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a4367-ae09-40c6-aa6d-c378b4fd001b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:30.604832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:30.674034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:30.674034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:30.674034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:30.674034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] PATH_INFO: `/v3/projects/41726e6bc1854b9da0661168174e8776` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:30.682376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41726e6bc1854b9da0661168174e8776)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:30.682651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:30.746918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41726e6bc1854b9da0661168174e8776', 'request_id': 'req-1b0b5cdd-18d3-461f-884e-92be00ff4c68'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.755014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41726e6bc1854b9da0661168174e8776', 'request_id': 'req-1b0b5cdd-18d3-461f-884e-92be00ff4c68'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.766272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41726e6bc1854b9da0661168174e8776', 'request_id': 'req-1b0b5cdd-18d3-461f-884e-92be00ff4c68'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.781360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] The token cache is being invalidate because project 41726e6bc1854b9da0661168174e8776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:30.781643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.782356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1b0b5cdd-18d3-461f-884e-92be00ff4c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:30.784620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 913/1826] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:30 2022] DELETE /identity/v3/projects/41726e6bc1854b9da0661168174e8776 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:30.792087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40094a15-9bdf-41c5-8368-e00e7dce104c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:30.794773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40094a15-9bdf-41c5-8368-e00e7dce104c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:30.795080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40094a15-9bdf-41c5-8368-e00e7dce104c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:30.819139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-40094a15-9bdf-41c5-8368-e00e7dce104c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:30.820728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-40094a15-9bdf-41c5-8368-e00e7dce104c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:30.899342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 914/1827] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:30.909713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538adad-ec2a-4f6c-9624-508a7cc83738 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:30.909713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538adad-ec2a-4f6c-9624-508a7cc83738 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:30.909867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538adad-ec2a-4f6c-9624-508a7cc83738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:30.933744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0538adad-ec2a-4f6c-9624-508a7cc83738 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:30.935320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0538adad-ec2a-4f6c-9624-508a7cc83738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:30.951256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40094a15-9bdf-41c5-8368-e00e7dce104c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:30.952331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 914/1828] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:30.965711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40094a15-9bdf-41c5-8368-e00e7dce104c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:30.972431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:30.973440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:30.974508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:30.974992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:30.976355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:30.976891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:30.978795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:30.997051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 915/1829] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:26:30.998858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538adad-ec2a-4f6c-9624-508a7cc83738 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.008661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538adad-ec2a-4f6c-9624-508a7cc83738 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.016630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b06191-925c-4456-9dd5-d6b3669d0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.017302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06191-925c-4456-9dd5-d6b3669d0df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.017580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06191-925c-4456-9dd5-d6b3669d0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.017845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06191-925c-4456-9dd5-d6b3669d0df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.018522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b06191-925c-4456-9dd5-d6b3669d0df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.020935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b06191-925c-4456-9dd5-d6b3669d0df1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.059685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 915/1830] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:30 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:31.067095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.092567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0e860-7447-46af-a1cd-e2f26d7459e5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.160100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6894453-2944-458c-8541-778aeb9b6f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.161189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6894453-2944-458c-8541-778aeb9b6f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.161527 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6894453-2944-458c-8541-778aeb9b6f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.161864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6894453-2944-458c-8541-778aeb9b6f80 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.162542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6894453-2944-458c-8541-778aeb9b6f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.164486 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6894453-2944-458c-8541-778aeb9b6f80 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.188602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 916/1831] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:31 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:31.192906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b06191-925c-4456-9dd5-d6b3669d0df1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.208757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b06191-925c-4456-9dd5-d6b3669d0df1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.214331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1783f-4073-49e6-a267-e025e378802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.214951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1783f-4073-49e6-a267-e025e378802e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.215229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1783f-4073-49e6-a267-e025e378802e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.215507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1783f-4073-49e6-a267-e025e378802e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.216180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1783f-4073-49e6-a267-e025e378802e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.217889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1783f-4073-49e6-a267-e025e378802e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.231463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 916/1832] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:31.235513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6894453-2944-458c-8541-778aeb9b6f80 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.251238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6894453-2944-458c-8541-778aeb9b6f80 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.263013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.263223 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.263589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.263928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.264659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.266423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.293730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 917/1833] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:31.302144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1783f-4073-49e6-a267-e025e378802e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.316921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1783f-4073-49e6-a267-e025e378802e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.323964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2bba0-3468-4fe0-872d-d3a916688719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.324637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2bba0-3468-4fe0-872d-d3a916688719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.325059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2bba0-3468-4fe0-872d-d3a916688719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.325338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2bba0-3468-4fe0-872d-d3a916688719 admin admin] PATH_INFO: `/v3/projects/cdc38b906d5a4422b8788c4d59b707a3/users/ce22844738a14d81b8899a2e9de56066/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.398264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2bba0-3468-4fe0-872d-d3a916688719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc38b906d5a4422b8788c4d59b707a3, user_id=ce22844738a14d81b8899a2e9de56066, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.401385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2bba0-3468-4fe0-872d-d3a916688719 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.419565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.421578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:31.430516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 917/1834] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:31 2022] PUT /identity/v3/projects/cdc38b906d5a4422b8788c4d59b707a3/users/ce22844738a14d81b8899a2e9de56066/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:31.443122 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e1d87-5cd7-42b3-a6c5-d85e0f263372 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.443446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e1d87-5cd7-42b3-a6c5-d85e0f263372 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.443782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e1d87-5cd7-42b3-a6c5-d85e0f263372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.445859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e128007-6a1e-46d9-bb98-baa08acf8a05 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.507928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6e1e1d87-5cd7-42b3-a6c5-d85e0f263372 None None] MFA Rules not processed for user `ce22844738a14d81b8899a2e9de56066`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:31.509821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6e1e1d87-5cd7-42b3-a6c5-d85e0f263372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:31.533157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:31.549017 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:31.550926 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.551654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.551928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.552234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] PATH_INFO: `/v3/users/7c0a732376c64dd4bbeb8f7e16a93fed` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.553387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0a732376c64dd4bbeb8f7e16a93fed)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.555891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.602040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 918/1835] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:31.603852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41633-ba63-4d24-b938-db912ce4ddd1 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.604161 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41633-ba63-4d24-b938-db912ce4ddd1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.604428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41633-ba63-4d24-b938-db912ce4ddd1 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.606101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 919/1836] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:26:31.609550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c41633-ba63-4d24-b938-db912ce4ddd1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.620072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0a732376c64dd4bbeb8f7e16a93fed'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:31.620923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0a732376c64dd4bbeb8f7e16a93fed'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:31.626966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c41633-ba63-4d24-b938-db912ce4ddd1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.630710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0a732376c64dd4bbeb8f7e16a93fed'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:31.633130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e1857-59ab-406a-9257-197a7f6b548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.633823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e1857-59ab-406a-9257-197a7f6b548e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.636054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e1857-59ab-406a-9257-197a7f6b548e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.636434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e1857-59ab-406a-9257-197a7f6b548e admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.637206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e1857-59ab-406a-9257-197a7f6b548e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.639506 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e1857-59ab-406a-9257-197a7f6b548e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.649176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0a732376c64dd4bbeb8f7e16a93fed'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:31.650634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d834098a-ce55-4e47-9568-838f266b52d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0a732376c64dd4bbeb8f7e16a93fed'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:31.674306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 918/1837] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:31 2022] DELETE /identity/v3/users/7c0a732376c64dd4bbeb8f7e16a93fed => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:31.676366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57560a-6630-4c55-a321-0cfb117a4589 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.676827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57560a-6630-4c55-a321-0cfb117a4589 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.677249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57560a-6630-4c55-a321-0cfb117a4589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.679117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 920/1838] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:31 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:31.681285 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37593c35-0e4d-4b22-96b5-35d59df1609a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.681763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37593c35-0e4d-4b22-96b5-35d59df1609a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.682229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37593c35-0e4d-4b22-96b5-35d59df1609a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.707803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-37593c35-0e4d-4b22-96b5-35d59df1609a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:31.709085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-37593c35-0e4d-4b22-96b5-35d59df1609a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:31.713971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9c57560a-6630-4c55-a321-0cfb117a4589 None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:31.715989 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9c57560a-6630-4c55-a321-0cfb117a4589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:31.784410 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 921/1839] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:31.785729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37593c35-0e4d-4b22-96b5-35d59df1609a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.790341 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-37593c35-0e4d-4b22-96b5-35d59df1609a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:31.804372 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37593c35-0e4d-4b22-96b5-35d59df1609a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.809559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa001f7c-cf6c-4b7a-9939-7ed4f31913d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.810173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa001f7c-cf6c-4b7a-9939-7ed4f31913d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.810484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa001f7c-cf6c-4b7a-9939-7ed4f31913d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.810760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa001f7c-cf6c-4b7a-9939-7ed4f31913d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.811457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa001f7c-cf6c-4b7a-9939-7ed4f31913d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.813327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa001f7c-cf6c-4b7a-9939-7ed4f31913d2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.813756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 919/1840] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:31.814561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57560a-6630-4c55-a321-0cfb117a4589 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.815870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9c57560a-6630-4c55-a321-0cfb117a4589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:31.839025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 922/1841] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:31 2022] POST /identity/v3/users => generated 288 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:31.840550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a5a01-4107-48c7-9898-b19bbfcb8270 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.840978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a5a01-4107-48c7-9898-b19bbfcb8270 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.841268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a5a01-4107-48c7-9898-b19bbfcb8270 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.842889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 923/1842] 10.222.0.16 () {54 vars in 961 bytes} [Mon Oct 17 10:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:26:31.843738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a5a01-4107-48c7-9898-b19bbfcb8270 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.860144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a5a01-4107-48c7-9898-b19bbfcb8270 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.865584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03bf87-ff79-4d6c-be7a-10c8198273bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.866290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03bf87-ff79-4d6c-be7a-10c8198273bd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.866898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03bf87-ff79-4d6c-be7a-10c8198273bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.867235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03bf87-ff79-4d6c-be7a-10c8198273bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.868395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03bf87-ff79-4d6c-be7a-10c8198273bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.869741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03bf87-ff79-4d6c-be7a-10c8198273bd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.882760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 924/1843] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:31.883079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7981a-6f34-4ce9-b727-d17819830d72 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.883368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7981a-6f34-4ce9-b727-d17819830d72 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.883639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7981a-6f34-4ce9-b727-d17819830d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.894796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57560a-6630-4c55-a321-0cfb117a4589 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.899100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0228e0-38ae-453c-a016-4e3869d819cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.899702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0228e0-38ae-453c-a016-4e3869d819cc service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.899973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0228e0-38ae-453c-a016-4e3869d819cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.900236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0228e0-38ae-453c-a016-4e3869d819cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.905923 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-93d7981a-6f34-4ce9-b727-d17819830d72 None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:31.907151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-93d7981a-6f34-4ce9-b727-d17819830d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:31.909812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0228e0-38ae-453c-a016-4e3869d819cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.911337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0228e0-38ae-453c-a016-4e3869d819cc service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.919694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 925/1844] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Oct 17 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:31.920674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7981a-6f34-4ce9-b727-d17819830d72 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.931834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7981a-6f34-4ce9-b727-d17819830d72 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.935373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 920/1845] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:31.937059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0228e0-38ae-453c-a016-4e3869d819cc service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.937237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ce997-23dd-47e0-b321-f214841e58c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.937930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce997-23dd-47e0-b321-f214841e58c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.938284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce997-23dd-47e0-b321-f214841e58c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.938659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce997-23dd-47e0-b321-f214841e58c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.939635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ce997-23dd-47e0-b321-f214841e58c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.941317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ce997-23dd-47e0-b321-f214841e58c1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.952280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0228e0-38ae-453c-a016-4e3869d819cc service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.954273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 926/1846] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:31.958725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839a4b0-fe5b-44fa-a146-029cd85c03ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.958909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ce997-23dd-47e0-b321-f214841e58c1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:31.959470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839a4b0-fe5b-44fa-a146-029cd85c03ee service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.959918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839a4b0-fe5b-44fa-a146-029cd85c03ee service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.961021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839a4b0-fe5b-44fa-a146-029cd85c03ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.970450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ce997-23dd-47e0-b321-f214841e58c1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:31.974210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839a4b0-fe5b-44fa-a146-029cd85c03ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:31.976059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b860-8b44-4ab5-b1fd-36ba5b9970f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:31.976272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839a4b0-fe5b-44fa-a146-029cd85c03ee service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:31.976966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b860-8b44-4ab5-b1fd-36ba5b9970f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:31.977348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b860-8b44-4ab5-b1fd-36ba5b9970f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:31.977717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b860-8b44-4ab5-b1fd-36ba5b9970f9 admin admin] PATH_INFO: `/v3/projects/6235c489d8044cf39b0b4456650471fd/users/a5ce9355ef324f2ba414c0cf7e0bf1ad/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:31.983347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:31.999222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0b860-8b44-4ab5-b1fd-36ba5b9970f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6235c489d8044cf39b0b4456650471fd, user_id=a5ce9355ef324f2ba414c0cf7e0bf1ad, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.002084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0b860-8b44-4ab5-b1fd-36ba5b9970f9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.003180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 921/1847] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:32.029941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 927/1848] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:31 2022] PUT /identity/v3/projects/6235c489d8044cf39b0b4456650471fd/users/a5ce9355ef324f2ba414c0cf7e0bf1ad/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:32.031496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecb10-cecb-4313-b861-04e9ea56e104 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.031878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecb10-cecb-4313-b861-04e9ea56e104 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.032516 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecb10-cecb-4313-b861-04e9ea56e104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.072453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7cdecb10-cecb-4313-b861-04e9ea56e104 None None] MFA Rules not processed for user `a5ce9355ef324f2ba414c0cf7e0bf1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:32.073745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7cdecb10-cecb-4313-b861-04e9ea56e104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.158083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 922/1849] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:32.163242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b860-8b44-4ab5-b1fd-36ba5b9970f9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.211428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:32.229501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78824f-41a7-4206-a49b-07d9a66481b5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.230055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78824f-41a7-4206-a49b-07d9a66481b5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.230368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78824f-41a7-4206-a49b-07d9a66481b5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.233655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 923/1850] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:26:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 10:26:32.242801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b860-8b44-4ab5-b1fd-36ba5b9970f9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:32.245237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0411b7-606e-4280-a2de-b4e26fecd15d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.245548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0411b7-606e-4280-a2de-b4e26fecd15d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.245931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0411b7-606e-4280-a2de-b4e26fecd15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.251993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:32.253349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.253810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.254245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.256848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.262620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.273229 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ed0411b7-606e-4280-a2de-b4e26fecd15d None None] MFA Rules not processed for user `dbb92e22612b400dbf390e149abbb105`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:32.274754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ed0411b7-606e-4280-a2de-b4e26fecd15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.309268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 928/1851] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:32 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:32.313434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.317362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.359101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 924/1852] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:32.360034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0411b7-606e-4280-a2de-b4e26fecd15d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.372364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0411b7-606e-4280-a2de-b4e26fecd15d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:32.386759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcf9f3-866d-4449-b520-4dce02aa2cae admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:32.392552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:32.393283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.393571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.393846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] PATH_INFO: `/v3/projects/c4cb0a2061864071b439139294498694` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.395003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4cb0a2061864071b439139294498694)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.396968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.445992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:32.448517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.448904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.449253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.449958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.451971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.454146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4cb0a2061864071b439139294498694', 'request_id': 'req-2f477fa2-327a-4b65-9abb-c9d954972c14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.462808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4cb0a2061864071b439139294498694', 'request_id': 'req-2f477fa2-327a-4b65-9abb-c9d954972c14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.473194 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4cb0a2061864071b439139294498694', 'request_id': 'req-2f477fa2-327a-4b65-9abb-c9d954972c14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.477059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 925/1853] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:32 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:32.478774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.479879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.489267 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] The token cache is being invalidate because project c4cb0a2061864071b439139294498694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:32.489695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.490574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.492252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 929/1854] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:32 2022] DELETE /identity/v3/projects/c4cb0a2061864071b439139294498694 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:32.492400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600e113-1c60-4ef7-bdff-0a3c2b0a2de1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:32.495343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.498040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.502263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:32.582201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:32.582818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.583257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.583568 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] PATH_INFO: `/v3/users/c8e673c433af430faa62d21fa124f0a6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.585986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e673c433af430faa62d21fa124f0a6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.587991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.621166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:32.642664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f477fa2-327a-4b65-9abb-c9d954972c14 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:32.650815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:32.651573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.651944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.652319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.655422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.670048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.672475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.678656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e673c433af430faa62d21fa124f0a6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.680097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e673c433af430faa62d21fa124f0a6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.684980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 930/1855] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:32.685530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.687124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.689245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e673c433af430faa62d21fa124f0a6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.701250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302dfa3-fcd4-4c65-8035-c369cf430ac6 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:32.704272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e673c433af430faa62d21fa124f0a6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.704967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3ea0e83-242a-4760-8b0b-19a4fd442649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e673c433af430faa62d21fa124f0a6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:32.706729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:32.707308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.707586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.707856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.708499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.710095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.718684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 926/1856] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:32 2022] DELETE /identity/v3/users/c8e673c433af430faa62d21fa124f0a6 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:32.721688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81043459-38ff-45ef-a8f2-41071c832ddf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.721688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81043459-38ff-45ef-a8f2-41071c832ddf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.721688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81043459-38ff-45ef-a8f2-41071c832ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.722620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 931/1857] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:32.727189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.729205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.743759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0498843-e58c-4486-8a7d-fdb395eb4781 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:32.752326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-81043459-38ff-45ef-a8f2-41071c832ddf None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:32.754390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-81043459-38ff-45ef-a8f2-41071c832ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.816758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af58a4-5fd3-468c-83b4-2b0d8ed8a385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:32.817629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af58a4-5fd3-468c-83b4-2b0d8ed8a385 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.817905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af58a4-5fd3-468c-83b4-2b0d8ed8a385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.818171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af58a4-5fd3-468c-83b4-2b0d8ed8a385 admin admin] PATH_INFO: `/v3/projects/8ae297ed5d6f468ca5766797e23223ee/users/3395029f8f374ee9b2ba3776724604f4/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.839688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af58a4-5fd3-468c-83b4-2b0d8ed8a385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae297ed5d6f468ca5766797e23223ee, user_id=3395029f8f374ee9b2ba3776724604f4, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.842780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af58a4-5fd3-468c-83b4-2b0d8ed8a385 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.858627 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 927/1858] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:32.861816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6840a-8f06-4d29-8548-7667bf94985e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.862099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6840a-8f06-4d29-8548-7667bf94985e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.863145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6840a-8f06-4d29-8548-7667bf94985e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.875439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 932/1859] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:32 2022] PUT /identity/v3/projects/8ae297ed5d6f468ca5766797e23223ee/users/3395029f8f374ee9b2ba3776724604f4/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:32.877054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af58a4-5fd3-468c-83b4-2b0d8ed8a385 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.895423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eba6840a-8f06-4d29-8548-7667bf94985e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:32.896904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eba6840a-8f06-4d29-8548-7667bf94985e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:32.936506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:32.967459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af58a4-5fd3-468c-83b4-2b0d8ed8a385 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:32.972393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18a628-1838-47b8-aa9c-053af69aa04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:32.973111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18a628-1838-47b8-aa9c-053af69aa04b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:32.973296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18a628-1838-47b8-aa9c-053af69aa04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:32.973544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18a628-1838-47b8-aa9c-053af69aa04b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:32.974190 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18a628-1838-47b8-aa9c-053af69aa04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:32.976139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18a628-1838-47b8-aa9c-053af69aa04b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:32.988907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 928/1860] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:32.989400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 933/1861] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:32.990505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18a628-1838-47b8-aa9c-053af69aa04b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.994558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6840a-8f06-4d29-8548-7667bf94985e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:32.996218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eba6840a-8f06-4d29-8548-7667bf94985e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.006801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18a628-1838-47b8-aa9c-053af69aa04b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.013919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.014561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.015879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.016292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.017162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:33.017476 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.018650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6840a-8f06-4d29-8548-7667bf94985e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.019287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.025316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.025944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.026220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.026531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] PATH_INFO: `/v3/projects/667dc2dbb03d448897690e87efde3898` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.028242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=667dc2dbb03d448897690e87efde3898)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.030637 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.033062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 934/1862] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:26:33.034216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.046166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd440a4-45bc-493f-b9be-59255af71967 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.055412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5e196-0746-45e7-a7ca-c3465cbf3c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.055627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5e196-0746-45e7-a7ca-c3465cbf3c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.055982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5e196-0746-45e7-a7ca-c3465cbf3c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.056364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5e196-0746-45e7-a7ca-c3465cbf3c5b admin admin] PATH_INFO: `/v3/projects/8ae297ed5d6f468ca5766797e23223ee/users/3395029f8f374ee9b2ba3776724604f4/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.059128 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5e196-0746-45e7-a7ca-c3465cbf3c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae297ed5d6f468ca5766797e23223ee, user_id=3395029f8f374ee9b2ba3776724604f4, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.062586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5e196-0746-45e7-a7ca-c3465cbf3c5b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.089493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 935/1863] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:33 2022] PUT /identity/v3/projects/8ae297ed5d6f468ca5766797e23223ee/users/3395029f8f374ee9b2ba3776724604f4/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:33.090556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5e196-0746-45e7-a7ca-c3465cbf3c5b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.103068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '667dc2dbb03d448897690e87efde3898', 'request_id': 'req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.109256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '667dc2dbb03d448897690e87efde3898', 'request_id': 'req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.115072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '667dc2dbb03d448897690e87efde3898', 'request_id': 'req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.121924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:33.129629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] The token cache is being invalidate because project 667dc2dbb03d448897690e87efde3898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:33.129917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.130600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-478a8a2f-a7d0-48d4-b7f3-23f1f2cf2cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.132663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 929/1864] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:32 2022] DELETE /identity/v3/projects/667dc2dbb03d448897690e87efde3898 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:33.133967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79ec27-d707-4930-9499-371b3f1d3507 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.134651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79ec27-d707-4930-9499-371b3f1d3507 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.134651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79ec27-d707-4930-9499-371b3f1d3507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.158337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5e196-0746-45e7-a7ca-c3465cbf3c5b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.162519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ce79ec27-d707-4930-9499-371b3f1d3507 None None] MFA Rules not processed for user `3395029f8f374ee9b2ba3776724604f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:33.164347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ce79ec27-d707-4930-9499-371b3f1d3507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.218835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.219441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.219713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.219979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.220604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.222517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.239004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 930/1865] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:33.245003 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79ec27-d707-4930-9499-371b3f1d3507 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.247084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ce79ec27-d707-4930-9499-371b3f1d3507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.252962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 936/1866] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:33 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:33.254147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.256220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.261753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79ec27-d707-4930-9499-371b3f1d3507 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.302653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:33.318205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5e243-cd0b-47f0-b1ab-6aff691a2225 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.322380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.322670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.322995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.323267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.323363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.323542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.323665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.323808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] PATH_INFO: `/v3/users/e0f70cf3fda94e6dbec0cca4c404553b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.324921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f70cf3fda94e6dbec0cca4c404553b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.325338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.326868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.375919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f70cf3fda94e6dbec0cca4c404553b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.376648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f70cf3fda94e6dbec0cca4c404553b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.384916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f70cf3fda94e6dbec0cca4c404553b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.388327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.389963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.396762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f70cf3fda94e6dbec0cca4c404553b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.397422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f70cf3fda94e6dbec0cca4c404553b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.399939 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 937/1867] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:33.400852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.402341 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.410876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 931/1868] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:33 2022] DELETE /identity/v3/users/e0f70cf3fda94e6dbec0cca4c404553b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:33.421572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6df3-0aaf-4a08-81a9-3f553decd3c9 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.437895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:33.509902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f71bb-c7ef-4ed5-abcc-3e5996b99e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.511615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f71bb-c7ef-4ed5-abcc-3e5996b99e43 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.512105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f71bb-c7ef-4ed5-abcc-3e5996b99e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.512636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f71bb-c7ef-4ed5-abcc-3e5996b99e43 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.513540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f71bb-c7ef-4ed5-abcc-3e5996b99e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.517515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f71bb-c7ef-4ed5-abcc-3e5996b99e43 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.543839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 938/1869] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:33 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:33.548118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.549703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.564153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d859dc0-f8b9-40a2-a8bd-872560b37f0e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.569013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313e33a-395e-4e71-b08d-13712b7d26ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.569736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313e33a-395e-4e71-b08d-13712b7d26ad admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.570086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313e33a-395e-4e71-b08d-13712b7d26ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.570383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313e33a-395e-4e71-b08d-13712b7d26ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.571406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313e33a-395e-4e71-b08d-13712b7d26ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.573016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313e33a-395e-4e71-b08d-13712b7d26ad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.588494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 932/1870] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:26:33.593168 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f71bb-c7ef-4ed5-abcc-3e5996b99e43 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.605407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f71bb-c7ef-4ed5-abcc-3e5996b99e43 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.611568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.612182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.612451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.612803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.613553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.615957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.634514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 939/1871] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:33.638077 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313e33a-395e-4e71-b08d-13712b7d26ad admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.641741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.643838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.651542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313e33a-395e-4e71-b08d-13712b7d26ad admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.667226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb94f1-e940-4546-9bdb-2329e55e8e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.668505 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb94f1-e940-4546-9bdb-2329e55e8e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.668971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb94f1-e940-4546-9bdb-2329e55e8e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.669549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb94f1-e940-4546-9bdb-2329e55e8e6e admin admin] PATH_INFO: `/v3/projects/cc21860aa70345c79cfe597d5a42fb51/users/973791f2609b4b55aa986e9fd9baceec/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.688151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f00968-f1cd-4b72-936d-df75ba8e56f5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:33.697286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:33.698013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.698438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.698825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] PATH_INFO: `/v3/projects/2b4d79715f514596a8e128bce2362603` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.700188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4d79715f514596a8e128bce2362603)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.703127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.709281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb94f1-e940-4546-9bdb-2329e55e8e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc21860aa70345c79cfe597d5a42fb51, user_id=973791f2609b4b55aa986e9fd9baceec, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:33.720266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb94f1-e940-4546-9bdb-2329e55e8e6e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:33.750903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 933/1872] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:33 2022] PUT /identity/v3/projects/cc21860aa70345c79cfe597d5a42fb51/users/973791f2609b4b55aa986e9fd9baceec/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:33.753153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816bacf-2208-4d60-ba9e-13bdee09c743 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.753653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816bacf-2208-4d60-ba9e-13bdee09c743 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.754233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816bacf-2208-4d60-ba9e-13bdee09c743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.785615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8816bacf-2208-4d60-ba9e-13bdee09c743 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:33.788214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8816bacf-2208-4d60-ba9e-13bdee09c743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.798668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4d79715f514596a8e128bce2362603', 'request_id': 'req-eee32d56-f563-4f63-aaed-245616401cf6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.821541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4d79715f514596a8e128bce2362603', 'request_id': 'req-eee32d56-f563-4f63-aaed-245616401cf6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.833166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4d79715f514596a8e128bce2362603', 'request_id': 'req-eee32d56-f563-4f63-aaed-245616401cf6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.856539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] The token cache is being invalidate because project 2b4d79715f514596a8e128bce2362603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:33.857023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.868786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eee32d56-f563-4f63-aaed-245616401cf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:33.880512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 940/1873] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:33 2022] DELETE /identity/v3/projects/2b4d79715f514596a8e128bce2362603 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:33.883490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef89170-f6a8-467a-a631-9922ab935592 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.883870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef89170-f6a8-467a-a631-9922ab935592 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.884242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef89170-f6a8-467a-a631-9922ab935592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.908352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6ef89170-f6a8-467a-a631-9922ab935592 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:33.910982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6ef89170-f6a8-467a-a631-9922ab935592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.915289 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 934/1874] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:33.923181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 941/1875] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:26:33.932265 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816bacf-2208-4d60-ba9e-13bdee09c743 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.932773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1845f-acbd-45f6-b976-685ffeb73059 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:33.933131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1845f-acbd-45f6-b976-685ffeb73059 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:33.933366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1845f-acbd-45f6-b976-685ffeb73059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:33.959100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8816bacf-2208-4d60-ba9e-13bdee09c743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.964599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9dc1845f-acbd-45f6-b976-685ffeb73059 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:33.966006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9dc1845f-acbd-45f6-b976-685ffeb73059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:33.979425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 942/1876] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:26:33.984200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1845f-acbd-45f6-b976-685ffeb73059 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:33.995430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1845f-acbd-45f6-b976-685ffeb73059 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:34.001542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:34.001829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:34.002292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.002677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.003037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.003705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:34.004057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:34.005652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:34.019384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 943/1877] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:26:34.023828 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:34.033762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17463d-a50c-4f13-8bb4-641309fdfd77 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:34.039485 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766632dc-28c7-46a8-b781-8ed534df4ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:34.040135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766632dc-28c7-46a8-b781-8ed534df4ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.040415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766632dc-28c7-46a8-b781-8ed534df4ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.040685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766632dc-28c7-46a8-b781-8ed534df4ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.041313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766632dc-28c7-46a8-b781-8ed534df4ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:34.043636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766632dc-28c7-46a8-b781-8ed534df4ee1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:34.045011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816bacf-2208-4d60-ba9e-13bdee09c743 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:34.048770 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76111b-5c9b-4145-a200-40f7df88386e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:34.049751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76111b-5c9b-4145-a200-40f7df88386e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.051402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76111b-5c9b-4145-a200-40f7df88386e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.051402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76111b-5c9b-4145-a200-40f7df88386e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.064567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76111b-5c9b-4145-a200-40f7df88386e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:34.066085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76111b-5c9b-4145-a200-40f7df88386e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:34.077786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 944/1878] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:34 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:26:34.082850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766632dc-28c7-46a8-b781-8ed534df4ee1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:34.086231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 935/1879] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:34.123393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:34.159302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766632dc-28c7-46a8-b781-8ed534df4ee1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:34.165840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211be5-759c-4443-bff4-58585f2d0e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:34.166636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211be5-759c-4443-bff4-58585f2d0e53 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.166912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211be5-759c-4443-bff4-58585f2d0e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.167188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211be5-759c-4443-bff4-58585f2d0e53 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.168284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2211be5-759c-4443-bff4-58585f2d0e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:34.170527 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2211be5-759c-4443-bff4-58585f2d0e53 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:34.198241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 945/1880] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:34 2022] POST /identity/v3/users => generated 282 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:26:34.202773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76111b-5c9b-4145-a200-40f7df88386e service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:34.214147 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76111b-5c9b-4145-a200-40f7df88386e service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:34.219216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb309d-5f8c-4dec-95ee-fbb92215dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:34.219820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb309d-5f8c-4dec-95ee-fbb92215dab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.220093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb309d-5f8c-4dec-95ee-fbb92215dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.220372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb309d-5f8c-4dec-95ee-fbb92215dab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.221025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb309d-5f8c-4dec-95ee-fbb92215dab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:34.222805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb309d-5f8c-4dec-95ee-fbb92215dab9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:34.238313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 936/1881] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:26:34.245924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211be5-759c-4443-bff4-58585f2d0e53 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:34.259207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211be5-759c-4443-bff4-58585f2d0e53 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:34.264456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad48068-4ed0-4840-b02e-4e8353810ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:34.265151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad48068-4ed0-4840-b02e-4e8353810ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.265551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad48068-4ed0-4840-b02e-4e8353810ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.265943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad48068-4ed0-4840-b02e-4e8353810ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.266628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad48068-4ed0-4840-b02e-4e8353810ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:34.268373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad48068-4ed0-4840-b02e-4e8353810ad1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:34.280497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 946/1882] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:34.285149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb309d-5f8c-4dec-95ee-fbb92215dab9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:34.297140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb309d-5f8c-4dec-95ee-fbb92215dab9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:34.302259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:34.302953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.303241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.303497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] PATH_INFO: `/v3/projects/c45638677b3540e281a8c5771d1dca24/users/6985b102dfd84ed0aee49b3d420007ca/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.321498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45638677b3540e281a8c5771d1dca24, user_id=6985b102dfd84ed0aee49b3d420007ca, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:34.324521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:34.342184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 937/1883] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:34 2022] PUT /identity/v3/projects/c45638677b3540e281a8c5771d1dca24/users/6985b102dfd84ed0aee49b3d420007ca/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:34.353884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1297f8-67f7-47d0-a0e2-8afc51c0e396 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.354362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1297f8-67f7-47d0-a0e2-8afc51c0e396 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.354702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1297f8-67f7-47d0-a0e2-8afc51c0e396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.385278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ac1297f8-67f7-47d0-a0e2-8afc51c0e396 None None] MFA Rules not processed for user `6985b102dfd84ed0aee49b3d420007ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:34.386603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ac1297f8-67f7-47d0-a0e2-8afc51c0e396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:34.465223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 947/1884] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:34.473682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:34.476104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:34.542670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1cb4a-8675-4205-aa28-49ee924dcf8e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:34.546435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f97a3f-65c6-429d-9afe-b1b87903bb1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:34.547036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97a3f-65c6-429d-9afe-b1b87903bb1c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:34.547634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97a3f-65c6-429d-9afe-b1b87903bb1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:34.547634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97a3f-65c6-429d-9afe-b1b87903bb1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:34.558689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f97a3f-65c6-429d-9afe-b1b87903bb1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:34.560457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f97a3f-65c6-429d-9afe-b1b87903bb1c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:34.570849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 938/1885] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:35.269532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1297f8-67f7-47d0-a0e2-8afc51c0e396 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:35.271614 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ac1297f8-67f7-47d0-a0e2-8afc51c0e396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:35.295684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1297f8-67f7-47d0-a0e2-8afc51c0e396 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:35.303181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8da24-a41c-4425-89db-3e9eb0dfdea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:35.303955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8da24-a41c-4425-89db-3e9eb0dfdea7 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:35.304923 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8da24-a41c-4425-89db-3e9eb0dfdea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:35.305209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8da24-a41c-4425-89db-3e9eb0dfdea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:35.308592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dfb8da24-a41c-4425-89db-3e9eb0dfdea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:35.408157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8da24-a41c-4425-89db-3e9eb0dfdea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:35.409952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8da24-a41c-4425-89db-3e9eb0dfdea7 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:35.428542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 948/1886] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:36.468743 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f97a3f-65c6-429d-9afe-b1b87903bb1c service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:36.480431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f97a3f-65c6-429d-9afe-b1b87903bb1c service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:36.484920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:36.485612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:36.485973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:36.486247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:36.489094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:36.579769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:36.581375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:36.593866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 939/1887] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:40.140172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49dfeb-57f8-4074-ae71-c45693f877a1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:40.140804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49dfeb-57f8-4074-ae71-c45693f877a1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:40.140804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49dfeb-57f8-4074-ae71-c45693f877a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:40.177569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7b49dfeb-57f8-4074-ae71-c45693f877a1 None None] MFA Rules not processed for user `973791f2609b4b55aa986e9fd9baceec`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:40.179089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7b49dfeb-57f8-4074-ae71-c45693f877a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:40.205121 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:40.271122 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 949/1888] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:40.276402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:40.278217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:40.330753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:40.385869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b1d18-4022-43ff-aa58-b9684b45978c service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:40.395453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd32139-15e3-4fc6-985f-c71f7a84d8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:40.398666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd32139-15e3-4fc6-985f-c71f7a84d8ad admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:40.399880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd32139-15e3-4fc6-985f-c71f7a84d8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:40.400552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd32139-15e3-4fc6-985f-c71f7a84d8ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:40.402344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd32139-15e3-4fc6-985f-c71f7a84d8ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:40.409195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd32139-15e3-4fc6-985f-c71f7a84d8ad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:40.468407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 940/1889] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:40 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:26:40.475273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49dfeb-57f8-4074-ae71-c45693f877a1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:40.487764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49dfeb-57f8-4074-ae71-c45693f877a1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:40.551307 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478b200-4767-4960-bc3f-60863e36da3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:40.551928 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478b200-4767-4960-bc3f-60863e36da3d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:40.552201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478b200-4767-4960-bc3f-60863e36da3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:40.552469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478b200-4767-4960-bc3f-60863e36da3d admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:40.553106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478b200-4767-4960-bc3f-60863e36da3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:40.556464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478b200-4767-4960-bc3f-60863e36da3d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:40.589710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 950/1890] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:40 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:40.594084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd32139-15e3-4fc6-985f-c71f7a84d8ad admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:40.607086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd32139-15e3-4fc6-985f-c71f7a84d8ad admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:40.612221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3022ee-4db0-46e8-9f05-bd3bdf0ad84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:40.612789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3022ee-4db0-46e8-9f05-bd3bdf0ad84f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:40.613688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3022ee-4db0-46e8-9f05-bd3bdf0ad84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:40.613796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3022ee-4db0-46e8-9f05-bd3bdf0ad84f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:40.614744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3022ee-4db0-46e8-9f05-bd3bdf0ad84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:40.617307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3022ee-4db0-46e8-9f05-bd3bdf0ad84f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:40.638636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 941/1891] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:40.639648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478b200-4767-4960-bc3f-60863e36da3d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:40.653462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478b200-4767-4960-bc3f-60863e36da3d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:40.661456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495c1ab-50d4-4dbd-a604-f3f3a1535a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:40.662148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495c1ab-50d4-4dbd-a604-f3f3a1535a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:40.662522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495c1ab-50d4-4dbd-a604-f3f3a1535a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:40.662791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495c1ab-50d4-4dbd-a604-f3f3a1535a1c admin admin] PATH_INFO: `/v3/projects/2baaa5333cee4891a7d766231743d38e/users/39fa012345254c56b8abc88fd079022f/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:40.685786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495c1ab-50d4-4dbd-a604-f3f3a1535a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2baaa5333cee4891a7d766231743d38e, user_id=39fa012345254c56b8abc88fd079022f, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:40.689606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495c1ab-50d4-4dbd-a604-f3f3a1535a1c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:40.705992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 951/1892] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:40 2022] PUT /identity/v3/projects/2baaa5333cee4891a7d766231743d38e/users/39fa012345254c56b8abc88fd079022f/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:40.711482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3022ee-4db0-46e8-9f05-bd3bdf0ad84f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:40.724354 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3022ee-4db0-46e8-9f05-bd3bdf0ad84f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:40.802387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:40.803947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:40.804708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:40.805269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:40.806358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:40.808957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:40.826280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 942/1893] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:26:40.828179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495c1ab-50d4-4dbd-a604-f3f3a1535a1c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:40.848363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495c1ab-50d4-4dbd-a604-f3f3a1535a1c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:40.857275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f94c-53ff-4825-adf0-c68df4107f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:40.858364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f94c-53ff-4825-adf0-c68df4107f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:40.858842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f94c-53ff-4825-adf0-c68df4107f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:40.859393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f94c-53ff-4825-adf0-c68df4107f63 admin admin] PATH_INFO: `/v3/projects/2baaa5333cee4891a7d766231743d38e/users/39fa012345254c56b8abc88fd079022f/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:40.861700 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797f94c-53ff-4825-adf0-c68df4107f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2baaa5333cee4891a7d766231743d38e, user_id=39fa012345254c56b8abc88fd079022f, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:40.867663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797f94c-53ff-4825-adf0-c68df4107f63 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:40.888164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 952/1894] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:40 2022] PUT /identity/v3/projects/2baaa5333cee4891a7d766231743d38e/users/39fa012345254c56b8abc88fd079022f/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:43.352697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:43.355629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:43.376245 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f59e29-3a42-4cec-baea-da5fa054004c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:43.474707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:43.475693 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:43.476768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:43.477740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] PATH_INFO: `/v3/users/ce22844738a14d81b8899a2e9de56066` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:43.479472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce22844738a14d81b8899a2e9de56066)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:43.482106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:43.600031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:43.607721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce22844738a14d81b8899a2e9de56066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:43.610118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce22844738a14d81b8899a2e9de56066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:43.629564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce22844738a14d81b8899a2e9de56066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:43.655493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce22844738a14d81b8899a2e9de56066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:43.656744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-705fdd31-3613-4543-9ee0-f62ad5fa6850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce22844738a14d81b8899a2e9de56066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:43.693616 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902dd11-5924-48d0-955d-632ccd3ff709 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:43.694098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 943/1895] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:43 2022] DELETE /identity/v3/users/ce22844738a14d81b8899a2e9de56066 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:43.695221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902dd11-5924-48d0-955d-632ccd3ff709 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:43.695551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902dd11-5924-48d0-955d-632ccd3ff709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:43.734796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1902dd11-5924-48d0-955d-632ccd3ff709 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:43.736170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1902dd11-5924-48d0-955d-632ccd3ff709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:43.745839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2dc72-d141-4988-925a-1f520720dc59 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:43.746125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2dc72-d141-4988-925a-1f520720dc59 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:43.754299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2dc72-d141-4988-925a-1f520720dc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:43.800197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9ef2dc72-d141-4988-925a-1f520720dc59 None None] MFA Rules not processed for user `a5b7190649694b9882ab34c0f3114a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:43.802060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9ef2dc72-d141-4988-925a-1f520720dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:43.924211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 953/1896] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:26:43.946068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902dd11-5924-48d0-955d-632ccd3ff709 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:43.955554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1902dd11-5924-48d0-955d-632ccd3ff709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:43.957102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 944/1897] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:43.999455 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:44.083403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902dd11-5924-48d0-955d-632ccd3ff709 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:44.088340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:44.089067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:44.089353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:44.089626 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:44.105787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:44.107363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:44.124840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 954/1898] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:26:44.983711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2dc72-d141-4988-925a-1f520720dc59 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:44.987313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9ef2dc72-d141-4988-925a-1f520720dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:45.014474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2dc72-d141-4988-925a-1f520720dc59 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:45.024513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:45.025405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:45.025860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:45.026285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] PATH_INFO: `/v3/users/ca96437c5fdc4c1ba41ca7960813dfae` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:45.027777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca96437c5fdc4c1ba41ca7960813dfae)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:45.029972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:45.145157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca96437c5fdc4c1ba41ca7960813dfae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.156569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca96437c5fdc4c1ba41ca7960813dfae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.177035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca96437c5fdc4c1ba41ca7960813dfae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.196140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca96437c5fdc4c1ba41ca7960813dfae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.197374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca96437c5fdc4c1ba41ca7960813dfae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.215645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 945/1899] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:44 2022] DELETE /identity/v3/users/ca96437c5fdc4c1ba41ca7960813dfae => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:45.461206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:45.462734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:45.492816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770da73f-c84b-48c9-a5f6-d481dd9511d8 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:45.675638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:45.676286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:45.676565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:45.676834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] PATH_INFO: `/v3/projects/cdc38b906d5a4422b8788c4d59b707a3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:45.677990 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc38b906d5a4422b8788c4d59b707a3)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:45.694032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:45.791286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc38b906d5a4422b8788c4d59b707a3', 'request_id': 'req-bed3ca42-0fc3-44d7-8122-85602bbf8b61'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.808347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc38b906d5a4422b8788c4d59b707a3', 'request_id': 'req-bed3ca42-0fc3-44d7-8122-85602bbf8b61'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.827196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc38b906d5a4422b8788c4d59b707a3', 'request_id': 'req-bed3ca42-0fc3-44d7-8122-85602bbf8b61'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.845215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] The token cache is being invalidate because project cdc38b906d5a4422b8788c4d59b707a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:45.845517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.846579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:45.857695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 955/1900] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:45 2022] DELETE /identity/v3/projects/cdc38b906d5a4422b8788c4d59b707a3 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:45.858145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:45.861968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:45.903562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa873b4-2df9-465e-985f-ea73c4084b5e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:45.995073 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:46.050699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:46.052506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:46.053040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:46.058780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] PATH_INFO: `/v3/users/a5ce9355ef324f2ba414c0cf7e0bf1ad` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:46.060366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ce9355ef324f2ba414c0cf7e0bf1ad)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:46.062860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:46.183299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ce9355ef324f2ba414c0cf7e0bf1ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.184807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ce9355ef324f2ba414c0cf7e0bf1ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.207517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ce9355ef324f2ba414c0cf7e0bf1ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.221431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:46.222956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:46.239101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ce9355ef324f2ba414c0cf7e0bf1ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.241020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3a02f18-d951-49b8-b521-c8299cde3df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ce9355ef324f2ba414c0cf7e0bf1ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.261252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 946/1901] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:45 2022] DELETE /identity/v3/users/a5ce9355ef324f2ba414c0cf7e0bf1ad => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:46.286365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3ca42-0fc3-44d7-8122-85602bbf8b61 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:46.422251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:46.509199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:46.509994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:46.510365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:46.510996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] PATH_INFO: `/v3/projects/6e2cba05193f429fb9ffef6f66205945` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:46.516123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2cba05193f429fb9ffef6f66205945)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:46.518804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:46.652199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2cba05193f429fb9ffef6f66205945', 'request_id': 'req-9265202c-d174-485f-9856-d0ab99657965'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.669251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2cba05193f429fb9ffef6f66205945', 'request_id': 'req-9265202c-d174-485f-9856-d0ab99657965'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.687966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2cba05193f429fb9ffef6f66205945', 'request_id': 'req-9265202c-d174-485f-9856-d0ab99657965'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.707333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] The token cache is being invalidate because project 6e2cba05193f429fb9ffef6f66205945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:46.707783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.710957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:46.721822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b480-3ba5-46bb-8b17-eb55f17aa239 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:46.722037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b480-3ba5-46bb-8b17-eb55f17aa239 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:46.722304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b480-3ba5-46bb-8b17-eb55f17aa239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:46.729722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 956/1902] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:46 2022] DELETE /identity/v3/projects/6e2cba05193f429fb9ffef6f66205945 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:46.798138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-79a2b480-3ba5-46bb-8b17-eb55f17aa239 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:46.803451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-79a2b480-3ba5-46bb-8b17-eb55f17aa239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:46.916844 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:46.918770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:46.986726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 947/1903] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:46.993388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7cff1-af4d-4feb-a7fe-918f6d72160a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:46.993883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7cff1-af4d-4feb-a7fe-918f6d72160a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:46.994389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7cff1-af4d-4feb-a7fe-918f6d72160a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.077898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-14f7cff1-af4d-4feb-a7fe-918f6d72160a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:47.079171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-14f7cff1-af4d-4feb-a7fe-918f6d72160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:47.105393 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 948/1904] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:47.109571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7cff1-af4d-4feb-a7fe-918f6d72160a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:47.129656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7cff1-af4d-4feb-a7fe-918f6d72160a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:47.131325 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265202c-d174-485f-9856-d0ab99657965 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:47.137560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:47.138430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.139013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.139618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.140525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:47.141063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:47.143250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:47.145711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:47.146400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.146682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.146951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] PATH_INFO: `/v3/projects/6235c489d8044cf39b0b4456650471fd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.151206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6235c489d8044cf39b0b4456650471fd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:47.153174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:47.160165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 949/1905] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:26:47.165312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:47.181079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3ae8-7571-48f1-aa5d-4aae343e0ffa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:47.188474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4340-7fff-4ea2-a57e-3ad9c8ef9169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:47.189383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4340-7fff-4ea2-a57e-3ad9c8ef9169 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.189887 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4340-7fff-4ea2-a57e-3ad9c8ef9169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.191460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4340-7fff-4ea2-a57e-3ad9c8ef9169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.193046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d4340-7fff-4ea2-a57e-3ad9c8ef9169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:47.196312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d4340-7fff-4ea2-a57e-3ad9c8ef9169 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:47.243916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 950/1906] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:47 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:47.245609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4340-7fff-4ea2-a57e-3ad9c8ef9169 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:47.255661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6235c489d8044cf39b0b4456650471fd', 'request_id': 'req-3f9e7157-c6b8-45b1-a016-0989af8988ae'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.269607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6235c489d8044cf39b0b4456650471fd', 'request_id': 'req-3f9e7157-c6b8-45b1-a016-0989af8988ae'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.275036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:47.285631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6235c489d8044cf39b0b4456650471fd', 'request_id': 'req-3f9e7157-c6b8-45b1-a016-0989af8988ae'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.305281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] The token cache is being invalidate because project 6235c489d8044cf39b0b4456650471fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:47.305704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.306369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.314274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 957/1907] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:46 2022] DELETE /identity/v3/projects/6235c489d8044cf39b0b4456650471fd => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:47.318930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:47.322284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:47.357791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:47.400145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4340-7fff-4ea2-a57e-3ad9c8ef9169 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:47.419934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:47.422156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.423694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.424135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.425061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:47.432118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:47.485609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 951/1908] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:47 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:47.494602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:47.501066 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:47.519996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e7157-c6b8-45b1-a016-0989af8988ae admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:47.536993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca6266-620c-4597-add8-6d118d63fd1f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:47.544787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f6f34-8f9c-44fd-a22f-1cd87946412d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:47.545483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f6f34-8f9c-44fd-a22f-1cd87946412d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.546019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f6f34-8f9c-44fd-a22f-1cd87946412d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.546297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f6f34-8f9c-44fd-a22f-1cd87946412d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.547000 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:47.547221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f6f34-8f9c-44fd-a22f-1cd87946412d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:47.548831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f6f34-8f9c-44fd-a22f-1cd87946412d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:47.549383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.549804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.550270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] PATH_INFO: `/v3/users/3395029f8f374ee9b2ba3776724604f4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.553618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3395029f8f374ee9b2ba3776724604f4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:47.556610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:47.562936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 952/1909] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:47.566933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f6f34-8f9c-44fd-a22f-1cd87946412d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:47.582885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f6f34-8f9c-44fd-a22f-1cd87946412d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:47.593514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd4a4d-a42e-4a47-a675-1e7163c91d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:47.594493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4a4d-a42e-4a47-a675-1e7163c91d57 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.595236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4a4d-a42e-4a47-a675-1e7163c91d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.596240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4a4d-a42e-4a47-a675-1e7163c91d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.597341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd4a4d-a42e-4a47-a675-1e7163c91d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:47.600591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd4a4d-a42e-4a47-a675-1e7163c91d57 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:47.616531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 953/1910] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:47.621537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd4a4d-a42e-4a47-a675-1e7163c91d57 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:47.639464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd4a4d-a42e-4a47-a675-1e7163c91d57 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:47.642277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3395029f8f374ee9b2ba3776724604f4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.643526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3395029f8f374ee9b2ba3776724604f4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.653558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce258297-0f00-452d-afc8-4a46164bd2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:47.654755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce258297-0f00-452d-afc8-4a46164bd2d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.655214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce258297-0f00-452d-afc8-4a46164bd2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.655762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce258297-0f00-452d-afc8-4a46164bd2d1 admin admin] PATH_INFO: `/v3/projects/0c2488861a034d2b8285e50ae6212aad/users/ef2bd54d094948c69d04cb62f14c9a8a/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.679002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce258297-0f00-452d-afc8-4a46164bd2d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2488861a034d2b8285e50ae6212aad, user_id=ef2bd54d094948c69d04cb62f14c9a8a, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:47.679229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3395029f8f374ee9b2ba3776724604f4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.682520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce258297-0f00-452d-afc8-4a46164bd2d1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:47.691283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3395029f8f374ee9b2ba3776724604f4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.693612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3395029f8f374ee9b2ba3776724604f4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:47.716318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 954/1911] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:47 2022] PUT /identity/v3/projects/0c2488861a034d2b8285e50ae6212aad/users/ef2bd54d094948c69d04cb62f14c9a8a/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:47.720816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e83de-76ec-416f-bd0f-fb8d3cdbcb75 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:47.721149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e83de-76ec-416f-bd0f-fb8d3cdbcb75 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:47.721429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e83de-76ec-416f-bd0f-fb8d3cdbcb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:47.737079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 958/1912] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:47 2022] DELETE /identity/v3/users/3395029f8f374ee9b2ba3776724604f4 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:47.765796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5e1e83de-76ec-416f-bd0f-fb8d3cdbcb75 None None] MFA Rules not processed for user `ef2bd54d094948c69d04cb62f14c9a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:47.767595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e1e83de-76ec-416f-bd0f-fb8d3cdbcb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:47.888270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 955/1913] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:47.896249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:47.900169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:48.089293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045be3e1-d8a3-4d12-bd0d-c73394a235fb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:48.097373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:48.098666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:48.099036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:48.099405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:48.113320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:48.115470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:48.132818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 959/1914] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:48.400942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e83de-76ec-416f-bd0f-fb8d3cdbcb75 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:48.402443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e1e83de-76ec-416f-bd0f-fb8d3cdbcb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:48.455233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:48.457670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:48.482101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:48.516501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:48.549670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e83de-76ec-416f-bd0f-fb8d3cdbcb75 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:48.570082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bf45b-e9c2-4dc1-843a-9283c2156688 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:48.581067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:48.581932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:48.582222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:48.582521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] PATH_INFO: `/v3/users/ef2bd54d094948c69d04cb62f14c9a8a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:48.583724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:48.584016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2bd54d094948c69d04cb62f14c9a8a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:48.585256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:48.585842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:48.586400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] PATH_INFO: `/v3/projects/8ae297ed5d6f468ca5766797e23223ee` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:48.586553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:48.590489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae297ed5d6f468ca5766797e23223ee)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:48.593593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:48.715277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2bd54d094948c69d04cb62f14c9a8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.718184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2bd54d094948c69d04cb62f14c9a8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.728352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae297ed5d6f468ca5766797e23223ee', 'request_id': 'req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.742725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae297ed5d6f468ca5766797e23223ee', 'request_id': 'req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.749146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2bd54d094948c69d04cb62f14c9a8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.752796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae297ed5d6f468ca5766797e23223ee', 'request_id': 'req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.769125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2bd54d094948c69d04cb62f14c9a8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.770012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52de6ee8-00c9-456f-ad20-bf323c83cd90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2bd54d094948c69d04cb62f14c9a8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.776385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] The token cache is being invalidate because project 8ae297ed5d6f468ca5766797e23223ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:48.776385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.785546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9113207b-7c75-4d5d-8a35-a85b2ffdfd45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:48.791414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 956/1915] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:48 2022] DELETE /identity/v3/projects/8ae297ed5d6f468ca5766797e23223ee => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:48.796740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7963fc-8ed0-46d7-8c0f-88fc42521f1c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:48.797423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7963fc-8ed0-46d7-8c0f-88fc42521f1c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:48.797759 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7963fc-8ed0-46d7-8c0f-88fc42521f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:48.799625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 960/1916] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:48 2022] DELETE /identity/v3/users/ef2bd54d094948c69d04cb62f14c9a8a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:26:48.801130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b4b81-2db5-497c-8676-9b8147e88c15 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:48.801624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b4b81-2db5-497c-8676-9b8147e88c15 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:48.801922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b4b81-2db5-497c-8676-9b8147e88c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:48.827167 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9b3b4b81-2db5-497c-8676-9b8147e88c15 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:48.828575 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9b3b4b81-2db5-497c-8676-9b8147e88c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:48.856587 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5b7963fc-8ed0-46d7-8c0f-88fc42521f1c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:48.860641 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b7963fc-8ed0-46d7-8c0f-88fc42521f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:48.937539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 961/1917] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:48.938803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2553-706e-4e76-a35f-36311224e3a5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:48.939801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2553-706e-4e76-a35f-36311224e3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:48.940092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2553-706e-4e76-a35f-36311224e3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:48.961987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 957/1918] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:48.963335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7963fc-8ed0-46d7-8c0f-88fc42521f1c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:48.964904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b7963fc-8ed0-46d7-8c0f-88fc42521f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:48.989600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-812b2553-706e-4e76-a35f-36311224e3a5 None None] MFA Rules not processed for user `39fa012345254c56b8abc88fd079022f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:48.991503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-812b2553-706e-4e76-a35f-36311224e3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:49.017214 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:49.053061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7963fc-8ed0-46d7-8c0f-88fc42521f1c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.059871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654d550-2147-492f-8b28-b8c04164b8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.059871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654d550-2147-492f-8b28-b8c04164b8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.060206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654d550-2147-492f-8b28-b8c04164b8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.061406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654d550-2147-492f-8b28-b8c04164b8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.078296 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654d550-2147-492f-8b28-b8c04164b8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.080522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654d550-2147-492f-8b28-b8c04164b8d3 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.103428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 958/1919] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:49.112675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 962/1920] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:49.117469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bd785-7992-4820-9e8b-69f6aa5b444d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.117938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bd785-7992-4820-9e8b-69f6aa5b444d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.119035 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bd785-7992-4820-9e8b-69f6aa5b444d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.126101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b2553-706e-4e76-a35f-36311224e3a5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.128207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-812b2553-706e-4e76-a35f-36311224e3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:49.177297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-769bd785-7992-4820-9e8b-69f6aa5b444d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:49.181401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-769bd785-7992-4820-9e8b-69f6aa5b444d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:49.232022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 959/1921] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:49.236460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bd785-7992-4820-9e8b-69f6aa5b444d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.253419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bd785-7992-4820-9e8b-69f6aa5b444d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.265891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.266768 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.267161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.267562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.268358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:49.269151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.270607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.289231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 960/1922] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:26:49.292096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b2553-706e-4e76-a35f-36311224e3a5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.293620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.303251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:49.307211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ac934ed0122470cb489d6b324c664f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.308645 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.310073 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.310852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.314149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d061347-149f-48b0-966c-2370b05b2fd0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.315680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:49.333584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636e45-34c6-42a4-ba0d-777c422a21e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.334510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636e45-34c6-42a4-ba0d-777c422a21e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.335652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636e45-34c6-42a4-ba0d-777c422a21e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.336305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636e45-34c6-42a4-ba0d-777c422a21e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.338172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636e45-34c6-42a4-ba0d-777c422a21e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.340338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636e45-34c6-42a4-ba0d-777c422a21e1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.412896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 961/1923] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:49 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:26:49.417247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636e45-34c6-42a4-ba0d-777c422a21e1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.437611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.438809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.450786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 963/1924] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:49.555619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636e45-34c6-42a4-ba0d-777c422a21e1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.559349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59312c5-7ff6-4675-82a8-189574b695a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.559983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59312c5-7ff6-4675-82a8-189574b695a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.560264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59312c5-7ff6-4675-82a8-189574b695a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.560538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59312c5-7ff6-4675-82a8-189574b695a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.561254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59312c5-7ff6-4675-82a8-189574b695a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.563322 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59312c5-7ff6-4675-82a8-189574b695a3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.591822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 962/1925] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:49 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:26:49.597744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.619649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5422a61-4ed6-4d0e-bb7e-88b620f9f280 service glance] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.627188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ee541-34ee-456c-ac74-9dde37923f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.630204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ee541-34ee-456c-ac74-9dde37923f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.631049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ee541-34ee-456c-ac74-9dde37923f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.631514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ee541-34ee-456c-ac74-9dde37923f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.632256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ee541-34ee-456c-ac74-9dde37923f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.634211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ee541-34ee-456c-ac74-9dde37923f88 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.646569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 964/1926] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:49.652351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59312c5-7ff6-4675-82a8-189574b695a3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.665534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e59b4a-5b83-4778-a54a-c64e333e6004 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.665819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e59b4a-5b83-4778-a54a-c64e333e6004 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.666095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e59b4a-5b83-4778-a54a-c64e333e6004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.669309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59312c5-7ff6-4675-82a8-189574b695a3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.676421 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.677109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.677391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.677713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.678538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.680254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.697408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f9e59b4a-5b83-4778-a54a-c64e333e6004 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:49.698902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f9e59b4a-5b83-4778-a54a-c64e333e6004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:49.699406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 963/1927] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:26:49.700495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.703070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:49.719185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9de1b-6206-49fe-a5f9-1d8133911044 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.724620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.726205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.726205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.726609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] PATH_INFO: `/v3/projects/0c2488861a034d2b8285e50ae6212aad` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.733276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2488861a034d2b8285e50ae6212aad)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.735428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.760136 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:49.783127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 965/1928] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:26:49.784471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e59b4a-5b83-4778-a54a-c64e333e6004 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.801657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2488861a034d2b8285e50ae6212aad', 'request_id': 'req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:49.805384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e59b4a-5b83-4778-a54a-c64e333e6004 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:49.814892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2488861a034d2b8285e50ae6212aad', 'request_id': 'req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:49.827801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2488861a034d2b8285e50ae6212aad', 'request_id': 'req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:49.844189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] The token cache is being invalidate because project 0c2488861a034d2b8285e50ae6212aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:49.844629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:49.845415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:49.847666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 964/1929] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:49 2022] DELETE /identity/v3/projects/0c2488861a034d2b8285e50ae6212aad => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:49.848350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:49.849805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:49.889463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37acfb-aabf-46c4-91db-c6fc4032879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:49.890205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37acfb-aabf-46c4-91db-c6fc4032879b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.890548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37acfb-aabf-46c4-91db-c6fc4032879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.890871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37acfb-aabf-46c4-91db-c6fc4032879b admin admin] PATH_INFO: `/v3/projects/eaf7678c1abd477d90c4062814f96762/users/0ef3c86766c8408cab29f2d221d88ec2/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.900778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:49.911538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37acfb-aabf-46c4-91db-c6fc4032879b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf7678c1abd477d90c4062814f96762, user_id=0ef3c86766c8408cab29f2d221d88ec2, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:49.915755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37acfb-aabf-46c4-91db-c6fc4032879b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:49.931284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 966/1930] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:49 2022] PUT /identity/v3/projects/eaf7678c1abd477d90c4062814f96762/users/0ef3c86766c8408cab29f2d221d88ec2/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:49.932958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f7a3b-b20b-4270-8220-65a30b763f26 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:49.933407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f7a3b-b20b-4270-8220-65a30b763f26 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:49.933717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f7a3b-b20b-4270-8220-65a30b763f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:49.943117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb09b8-94a2-4c19-96cd-1feb7d4298c2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:50.015742 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-af0f7a3b-b20b-4270-8220-65a30b763f26 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:50.017539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-af0f7a3b-b20b-4270-8220-65a30b763f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:50.065831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842199e-55f3-4278-97e0-63e2a6fb4e28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:50.066485 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842199e-55f3-4278-97e0-63e2a6fb4e28 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.066774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842199e-55f3-4278-97e0-63e2a6fb4e28 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.067097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842199e-55f3-4278-97e0-63e2a6fb4e28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.068968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7842199e-55f3-4278-97e0-63e2a6fb4e28 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:50.115899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 967/1931] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:50.118234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6a188-cd97-46aa-8247-ee5d73b989eb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.118772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6a188-cd97-46aa-8247-ee5d73b989eb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.119259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6a188-cd97-46aa-8247-ee5d73b989eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.159007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3cb6a188-cd97-46aa-8247-ee5d73b989eb None None] MFA Rules not processed for user `0ef3c86766c8408cab29f2d221d88ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:50.161086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3cb6a188-cd97-46aa-8247-ee5d73b989eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:50.183700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842199e-55f3-4278-97e0-63e2a6fb4e28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:50.186096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842199e-55f3-4278-97e0-63e2a6fb4e28 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:50.201574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 965/1932] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:26:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:26:50.204216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8021a-7084-4878-acf9-46464ecab3ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.204602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8021a-7084-4878-acf9-46464ecab3ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.204823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8021a-7084-4878-acf9-46464ecab3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.250523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-01d8021a-7084-4878-acf9-46464ecab3ad None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:50.251850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-01d8021a-7084-4878-acf9-46464ecab3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:50.263620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 966/1933] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:50.268411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8021a-7084-4878-acf9-46464ecab3ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:50.285371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8021a-7084-4878-acf9-46464ecab3ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:50.300897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:50.301524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.301823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.302096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.302791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:50.303158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:50.305650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:50.320662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:50.328436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 968/1934] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:50.331558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 967/1935] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:26:50.335539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6a188-cd97-46aa-8247-ee5d73b989eb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:50.343395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:50.345909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:50.358177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6a188-cd97-46aa-8247-ee5d73b989eb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:50.370886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981f59b-65c9-4cd1-88c9-bfd17b2896e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:50.371521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981f59b-65c9-4cd1-88c9-bfd17b2896e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.371796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981f59b-65c9-4cd1-88c9-bfd17b2896e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.372064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981f59b-65c9-4cd1-88c9-bfd17b2896e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.372710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981f59b-65c9-4cd1-88c9-bfd17b2896e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:50.376299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981f59b-65c9-4cd1-88c9-bfd17b2896e1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:50.420739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 969/1936] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:50 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:50.422259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981f59b-65c9-4cd1-88c9-bfd17b2896e1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:50.483769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd774b7-0461-4190-9d41-7e844b9c2b89 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:50.522916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:50.537163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981f59b-65c9-4cd1-88c9-bfd17b2896e1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:50.542737 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd9f5c-7142-4df7-a87a-581d8097fa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:50.543354 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd9f5c-7142-4df7-a87a-581d8097fa55 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.543632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd9f5c-7142-4df7-a87a-581d8097fa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.544494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd9f5c-7142-4df7-a87a-581d8097fa55 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.545146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd9f5c-7142-4df7-a87a-581d8097fa55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:50.547282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd9f5c-7142-4df7-a87a-581d8097fa55 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:50.600512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 970/1937] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:50 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:50.602553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd9f5c-7142-4df7-a87a-581d8097fa55 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:50.604012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa3400-2e66-48cf-9749-c06656834862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:50.605951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa3400-2e66-48cf-9749-c06656834862 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.606845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa3400-2e66-48cf-9749-c06656834862 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.607282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa3400-2e66-48cf-9749-c06656834862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.626253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd9f5c-7142-4df7-a87a-581d8097fa55 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:50.648583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02caa9d-8df9-4f75-9e1f-85a94ea4df7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:50.649513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02caa9d-8df9-4f75-9e1f-85a94ea4df7c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.650004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02caa9d-8df9-4f75-9e1f-85a94ea4df7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.650502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02caa9d-8df9-4f75-9e1f-85a94ea4df7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.654549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02caa9d-8df9-4f75-9e1f-85a94ea4df7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:50.656444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02caa9d-8df9-4f75-9e1f-85a94ea4df7c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:50.682782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 971/1938] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:50.687031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02caa9d-8df9-4f75-9e1f-85a94ea4df7c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:50.701468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02caa9d-8df9-4f75-9e1f-85a94ea4df7c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:50.704056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa3400-2e66-48cf-9749-c06656834862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:50.706055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa3400-2e66-48cf-9749-c06656834862 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:50.707392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a64f4a-b0ec-464b-8a45-4e66ec0c0518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:50.708337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a64f4a-b0ec-464b-8a45-4e66ec0c0518 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.708801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a64f4a-b0ec-464b-8a45-4e66ec0c0518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.709370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a64f4a-b0ec-464b-8a45-4e66ec0c0518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.710893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a64f4a-b0ec-464b-8a45-4e66ec0c0518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:50.713706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a64f4a-b0ec-464b-8a45-4e66ec0c0518 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:50.731337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 968/1939] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:26:50.755462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 972/1940] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:50.757792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa3400-2e66-48cf-9749-c06656834862 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:50.776227 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa3400-2e66-48cf-9749-c06656834862 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:50.781387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cec77e-95a6-4cb8-b5cf-24ef2f1d8d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:50.782056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec77e-95a6-4cb8-b5cf-24ef2f1d8d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.782351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec77e-95a6-4cb8-b5cf-24ef2f1d8d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.782650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec77e-95a6-4cb8-b5cf-24ef2f1d8d20 admin admin] PATH_INFO: `/v3/projects/f8212d6289834a9899dadb01d2f10e28/users/f06f2e1909cc47adab776215c7feab88/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.814276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cec77e-95a6-4cb8-b5cf-24ef2f1d8d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8212d6289834a9899dadb01d2f10e28, user_id=f06f2e1909cc47adab776215c7feab88, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:50.821058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cec77e-95a6-4cb8-b5cf-24ef2f1d8d20 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:50.863905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 969/1941] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:50 2022] PUT /identity/v3/projects/f8212d6289834a9899dadb01d2f10e28/users/f06f2e1909cc47adab776215c7feab88/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:50.868154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997297-8c73-42d8-8151-18a99392a35b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:50.868449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997297-8c73-42d8-8151-18a99392a35b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:50.871240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997297-8c73-42d8-8151-18a99392a35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:50.935357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e3997297-8c73-42d8-8151-18a99392a35b None None] MFA Rules not processed for user `f06f2e1909cc47adab776215c7feab88`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:50.939965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e3997297-8c73-42d8-8151-18a99392a35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:51.096405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 973/1942] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:51.105358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cec77e-95a6-4cb8-b5cf-24ef2f1d8d20 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:51.208931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cec77e-95a6-4cb8-b5cf-24ef2f1d8d20 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:51.216884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607838d-0cd7-4a60-b6b8-da6991bdea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:51.217501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607838d-0cd7-4a60-b6b8-da6991bdea3b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:51.217776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607838d-0cd7-4a60-b6b8-da6991bdea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:51.218090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607838d-0cd7-4a60-b6b8-da6991bdea3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:51.218784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607838d-0cd7-4a60-b6b8-da6991bdea3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:51.222247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607838d-0cd7-4a60-b6b8-da6991bdea3b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:51.304049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 970/1943] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:51.309080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3997297-8c73-42d8-8151-18a99392a35b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:51.361272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3997297-8c73-42d8-8151-18a99392a35b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:51.487994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:51.548037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83869053-982b-4204-a480-83b243a4f855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:51.548469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83869053-982b-4204-a480-83b243a4f855 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:51.549356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83869053-982b-4204-a480-83b243a4f855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:51.549635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83869053-982b-4204-a480-83b243a4f855 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:51.550497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83869053-982b-4204-a480-83b243a4f855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:51.554150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83869053-982b-4204-a480-83b243a4f855 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:51.619143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 974/1944] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:51 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:51.619840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607838d-0cd7-4a60-b6b8-da6991bdea3b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:51.631198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607838d-0cd7-4a60-b6b8-da6991bdea3b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:51.637355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b57b9f-f6b7-4b15-8af7-78aec27bd588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:51.638063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b57b9f-f6b7-4b15-8af7-78aec27bd588 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:51.638426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b57b9f-f6b7-4b15-8af7-78aec27bd588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:51.638801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b57b9f-f6b7-4b15-8af7-78aec27bd588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:51.639512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b57b9f-f6b7-4b15-8af7-78aec27bd588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:51.644503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b57b9f-f6b7-4b15-8af7-78aec27bd588 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:51.661211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 971/1945] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:26:51.662108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83869053-982b-4204-a480-83b243a4f855 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:51.676235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83869053-982b-4204-a480-83b243a4f855 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:51.682796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fd9db-7df6-4450-bfa2-60a8a6ada72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:51.683465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fd9db-7df6-4450-bfa2-60a8a6ada72c admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:51.683828 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fd9db-7df6-4450-bfa2-60a8a6ada72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:51.684100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fd9db-7df6-4450-bfa2-60a8a6ada72c admin admin] PATH_INFO: `/v3/projects/bd50088ea246418297404cf41b59e30a/users/8418a1d5fb464d4d8653a174ab71de3d/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:51.701448 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fd9db-7df6-4450-bfa2-60a8a6ada72c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd50088ea246418297404cf41b59e30a, user_id=8418a1d5fb464d4d8653a174ab71de3d, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:51.704285 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fd9db-7df6-4450-bfa2-60a8a6ada72c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:51.720232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 975/1946] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:51 2022] PUT /identity/v3/projects/bd50088ea246418297404cf41b59e30a/users/8418a1d5fb464d4d8653a174ab71de3d/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:51.724007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b57b9f-f6b7-4b15-8af7-78aec27bd588 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:51.737685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b57b9f-f6b7-4b15-8af7-78aec27bd588 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:51.770456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:51.822969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85049789-01c0-4fff-82d7-4a2e9b7d2b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:51.823910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049789-01c0-4fff-82d7-4a2e9b7d2b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:51.824257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049789-01c0-4fff-82d7-4a2e9b7d2b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:51.824598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049789-01c0-4fff-82d7-4a2e9b7d2b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:51.825321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85049789-01c0-4fff-82d7-4a2e9b7d2b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:51.827560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85049789-01c0-4fff-82d7-4a2e9b7d2b8a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:51.844246 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 972/1947] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:51.844995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fd9db-7df6-4450-bfa2-60a8a6ada72c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:51.862154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fd9db-7df6-4450-bfa2-60a8a6ada72c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:51.872818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff3722-e4fa-4b1b-9e35-dc8f4948f4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:51.873599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff3722-e4fa-4b1b-9e35-dc8f4948f4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:51.873883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff3722-e4fa-4b1b-9e35-dc8f4948f4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:51.874204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff3722-e4fa-4b1b-9e35-dc8f4948f4b4 admin admin] PATH_INFO: `/v3/projects/bd50088ea246418297404cf41b59e30a/users/8418a1d5fb464d4d8653a174ab71de3d/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:51.876253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff3722-e4fa-4b1b-9e35-dc8f4948f4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd50088ea246418297404cf41b59e30a, user_id=8418a1d5fb464d4d8653a174ab71de3d, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:51.878973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff3722-e4fa-4b1b-9e35-dc8f4948f4b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:51.906867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 976/1948] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:51 2022] PUT /identity/v3/projects/bd50088ea246418297404cf41b59e30a/users/8418a1d5fb464d4d8653a174ab71de3d/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:51.910212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818a0b-d6a0-49ca-985c-bf908588b343 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:51.910525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818a0b-d6a0-49ca-985c-bf908588b343 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:51.910798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818a0b-d6a0-49ca-985c-bf908588b343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:51.953944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-38818a0b-d6a0-49ca-985c-bf908588b343 None None] MFA Rules not processed for user `8418a1d5fb464d4d8653a174ab71de3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:51.955200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-38818a0b-d6a0-49ca-985c-bf908588b343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:52.103498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 973/1949] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:52.116144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff3722-e4fa-4b1b-9e35-dc8f4948f4b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:52.259886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff3722-e4fa-4b1b-9e35-dc8f4948f4b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:52.267123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:52.267904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:52.268185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:52.268453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:52.285514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:52.292361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:52.321045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 977/1950] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:52.485033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38818a0b-d6a0-49ca-985c-bf908588b343 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:52.489158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-38818a0b-d6a0-49ca-985c-bf908588b343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:52.504205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38818a0b-d6a0-49ca-985c-bf908588b343 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:52.509593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadadf39-0da0-456d-90a5-97297a4f506a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:52.511897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadadf39-0da0-456d-90a5-97297a4f506a service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:52.512385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadadf39-0da0-456d-90a5-97297a4f506a service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:52.512849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadadf39-0da0-456d-90a5-97297a4f506a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:52.642969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadadf39-0da0-456d-90a5-97297a4f506a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:52.649909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadadf39-0da0-456d-90a5-97297a4f506a service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:52.685476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 974/1951] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:53.748211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:53.751127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:53.855960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cb7d4-5c91-4506-a0ca-29546788f514 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:53.867196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:53.868117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:53.868701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:53.869177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] PATH_INFO: `/v3/users/0ef3c86766c8408cab29f2d221d88ec2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:53.870783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef3c86766c8408cab29f2d221d88ec2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:53.873715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:53.926198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:53.953867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef3c86766c8408cab29f2d221d88ec2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:53.955559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef3c86766c8408cab29f2d221d88ec2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:53.974278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef3c86766c8408cab29f2d221d88ec2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:54.007315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef3c86766c8408cab29f2d221d88ec2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:54.007315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef3c86766c8408cab29f2d221d88ec2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:54.037950 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 978/1952] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:26:53 2022] DELETE /identity/v3/users/0ef3c86766c8408cab29f2d221d88ec2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:54.043046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd96742-4573-49eb-bb2a-3f394cbdb96c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:54.043046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd96742-4573-49eb-bb2a-3f394cbdb96c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:54.043046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd96742-4573-49eb-bb2a-3f394cbdb96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:54.093026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ddd96742-4573-49eb-bb2a-3f394cbdb96c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:54.094704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ddd96742-4573-49eb-bb2a-3f394cbdb96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:54.205752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:54.265224 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 975/1953] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:54.275390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:54.276874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:54.409334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868628a-c2da-4243-a232-f6449fa6b469 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:54.420339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879b927-cf54-4d38-ad35-d4630abd5129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:54.421167 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b927-cf54-4d38-ad35-d4630abd5129 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:54.421467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b927-cf54-4d38-ad35-d4630abd5129 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:54.421790 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b927-cf54-4d38-ad35-d4630abd5129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:54.444727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879b927-cf54-4d38-ad35-d4630abd5129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:54.452859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879b927-cf54-4d38-ad35-d4630abd5129 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:54.512752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 979/1954] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:26:55.608900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd96742-4573-49eb-bb2a-3f394cbdb96c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:55.608900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ddd96742-4573-49eb-bb2a-3f394cbdb96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:55.625998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd96742-4573-49eb-bb2a-3f394cbdb96c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:55.648520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:55.649315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:55.649923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:55.650772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] PATH_INFO: `/v3/projects/eaf7678c1abd477d90c4062814f96762` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:55.653363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf7678c1abd477d90c4062814f96762)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:55.657543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:55.756257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf7678c1abd477d90c4062814f96762', 'request_id': 'req-7ac5ba20-6af6-4930-a597-a790f2ed619b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:55.764761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf7678c1abd477d90c4062814f96762', 'request_id': 'req-7ac5ba20-6af6-4930-a597-a790f2ed619b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:55.774919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf7678c1abd477d90c4062814f96762', 'request_id': 'req-7ac5ba20-6af6-4930-a597-a790f2ed619b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:55.799352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] The token cache is being invalidate because project eaf7678c1abd477d90c4062814f96762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:26:55.799727 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:55.800595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7ac5ba20-6af6-4930-a597-a790f2ed619b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:26:55.809190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 976/1955] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:26:55 2022] DELETE /identity/v3/projects/eaf7678c1abd477d90c4062814f96762 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:55.809513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01fa43-87e4-4561-97df-9d3451e761ff None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:55.810215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01fa43-87e4-4561-97df-9d3451e761ff None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:55.810714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01fa43-87e4-4561-97df-9d3451e761ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:55.852394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ed01fa43-87e4-4561-97df-9d3451e761ff None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:55.853886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ed01fa43-87e4-4561-97df-9d3451e761ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:55.947603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 980/1956] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:55.950072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd917d-d17b-4b19-9299-e61516e81bf3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:55.950724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd917d-d17b-4b19-9299-e61516e81bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:55.951242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd917d-d17b-4b19-9299-e61516e81bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:55.980014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-87dd917d-d17b-4b19-9299-e61516e81bf3 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:55.983133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-87dd917d-d17b-4b19-9299-e61516e81bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:56.004435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 977/1957] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:56.005968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01fa43-87e4-4561-97df-9d3451e761ff None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:56.026857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01fa43-87e4-4561-97df-9d3451e761ff None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:56.045994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:56.046651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:56.046926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:56.047195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:56.047920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:26:56.048225 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:56.049957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:56.081054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 981/1958] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:26:56.085291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd917d-d17b-4b19-9299-e61516e81bf3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:56.099025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd917d-d17b-4b19-9299-e61516e81bf3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:56.112011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31538bec-5b3a-4a14-a32f-ea7af742b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:56.113628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31538bec-5b3a-4a14-a32f-ea7af742b6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:56.114074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31538bec-5b3a-4a14-a32f-ea7af742b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:56.114649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31538bec-5b3a-4a14-a32f-ea7af742b6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:56.117194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31538bec-5b3a-4a14-a32f-ea7af742b6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:56.119551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31538bec-5b3a-4a14-a32f-ea7af742b6e5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:56.170233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 978/1959] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:26:56 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:56.174398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:56.192331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb98b-e890-4c7b-bf01-932cc252fa63 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:56.324757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:26:56.398086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80651c8a-2288-46c4-9751-85c4bcedc7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:56.399609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651c8a-2288-46c4-9751-85c4bcedc7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:56.402205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651c8a-2288-46c4-9751-85c4bcedc7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:56.403074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651c8a-2288-46c4-9751-85c4bcedc7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:56.404137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80651c8a-2288-46c4-9751-85c4bcedc7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:56.412516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80651c8a-2288-46c4-9751-85c4bcedc7f8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:56.480091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 982/1960] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:26:56 2022] POST /identity/v3/users => generated 286 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:26:56.481470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31538bec-5b3a-4a14-a32f-ea7af742b6e5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:56.500643 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31538bec-5b3a-4a14-a32f-ea7af742b6e5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:56.523244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c476b-442b-4091-a470-2f343afc897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:56.526935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c476b-442b-4091-a470-2f343afc897a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:56.527343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c476b-442b-4091-a470-2f343afc897a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:56.528124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c476b-442b-4091-a470-2f343afc897a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:56.528939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c476b-442b-4091-a470-2f343afc897a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:56.536220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c476b-442b-4091-a470-2f343afc897a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:56.577614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 979/1961] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:56.583166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80651c8a-2288-46c4-9751-85c4bcedc7f8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:56.596211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80651c8a-2288-46c4-9751-85c4bcedc7f8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:56.609443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11e877-fc96-44f4-8b5e-b44a07ea2a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:56.610197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11e877-fc96-44f4-8b5e-b44a07ea2a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:56.610599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11e877-fc96-44f4-8b5e-b44a07ea2a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:56.610963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11e877-fc96-44f4-8b5e-b44a07ea2a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:56.611805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11e877-fc96-44f4-8b5e-b44a07ea2a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:56.613751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11e877-fc96-44f4-8b5e-b44a07ea2a25 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:56.630268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 983/1962] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:26:56.632428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c476b-442b-4091-a470-2f343afc897a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:56.647428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c476b-442b-4091-a470-2f343afc897a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:56.667122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:56.667800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:56.668078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:56.668698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] PATH_INFO: `/v3/projects/0c02fbce467e4e86826cdab9719f9ef5/users/7f2cafc7b5ce4dd5a727f72843198876/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:56.704050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c02fbce467e4e86826cdab9719f9ef5, user_id=7f2cafc7b5ce4dd5a727f72843198876, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:56.708303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:56.786369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 980/1963] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:26:56 2022] PUT /identity/v3/projects/0c02fbce467e4e86826cdab9719f9ef5/users/7f2cafc7b5ce4dd5a727f72843198876/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:26:56.796985 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5499-c409-4693-9951-204aacb21bd4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:56.797522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5499-c409-4693-9951-204aacb21bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:56.797999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5499-c409-4693-9951-204aacb21bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:56.870753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-011a5499-c409-4693-9951-204aacb21bd4 None None] MFA Rules not processed for user `7f2cafc7b5ce4dd5a727f72843198876`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:26:56.873102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-011a5499-c409-4693-9951-204aacb21bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:57.088647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 984/1964] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:26:57.100072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:26:57.103653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:26:57.205335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9e598-5bab-4e49-b3b4-9814db76f63b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:26:57.216017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c13a-6ddc-4368-a280-db0053c6a9f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:26:57.216707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c13a-6ddc-4368-a280-db0053c6a9f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:26:57.217033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c13a-6ddc-4368-a280-db0053c6a9f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:26:57.217316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c13a-6ddc-4368-a280-db0053c6a9f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:26:57.242701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603c13a-6ddc-4368-a280-db0053c6a9f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:26:57.244745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603c13a-6ddc-4368-a280-db0053c6a9f1 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:26:57.255412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 981/1965] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:27:03.595117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919419e-ff85-4fc1-8d9b-270ce4723811 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:03.595760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919419e-ff85-4fc1-8d9b-270ce4723811 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:03.595760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919419e-ff85-4fc1-8d9b-270ce4723811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:03.628315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0919419e-ff85-4fc1-8d9b-270ce4723811 None None] MFA Rules not processed for user `973791f2609b4b55aa986e9fd9baceec`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:03.629500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0919419e-ff85-4fc1-8d9b-270ce4723811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:03.701841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 985/1966] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:06.549657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f503-db14-4314-a034-bf5818a8df9c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:06.550611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f503-db14-4314-a034-bf5818a8df9c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:06.551067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f503-db14-4314-a034-bf5818a8df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:06.580828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bf85f503-db14-4314-a034-bf5818a8df9c None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:06.582110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bf85f503-db14-4314-a034-bf5818a8df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:06.588220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:06.686838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 982/1967] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:27:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:06.699300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919419e-ff85-4fc1-8d9b-270ce4723811 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:06.700974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0919419e-ff85-4fc1-8d9b-270ce4723811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:06.781344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:06.838499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919419e-ff85-4fc1-8d9b-270ce4723811 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:06.850023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32688c-d6c7-4a54-98af-1e4756c6d6e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:06.850023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32688c-d6c7-4a54-98af-1e4756c6d6e4 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:06.850023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32688c-d6c7-4a54-98af-1e4756c6d6e4 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:06.850023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32688c-d6c7-4a54-98af-1e4756c6d6e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:06.856499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32688c-d6c7-4a54-98af-1e4756c6d6e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:06.858200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32688c-d6c7-4a54-98af-1e4756c6d6e4 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:06.877006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 986/1968] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:27:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:27:09.695472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6f45e-5464-451c-9c37-e97209adc3ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:09.696044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6f45e-5464-451c-9c37-e97209adc3ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:09.696044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6f45e-5464-451c-9c37-e97209adc3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:09.717412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-56f6f45e-5464-451c-9c37-e97209adc3ad None None] MFA Rules not processed for user `aaedb0e1920a42eb97e8dc2f3ec76905`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:09.719191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-56f6f45e-5464-451c-9c37-e97209adc3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:09.805415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 983/1969] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:12.376075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32688c-d6c7-4a54-98af-1e4756c6d6e4 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:12.455016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32688c-d6c7-4a54-98af-1e4756c6d6e4 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:12.460139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:12.460784 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:12.461057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:12.461479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] PATH_INFO: `/v3/users/7f2cafc7b5ce4dd5a727f72843198876` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:12.462687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2cafc7b5ce4dd5a727f72843198876)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:12.464609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:12.523055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2cafc7b5ce4dd5a727f72843198876'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:12.524211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2cafc7b5ce4dd5a727f72843198876'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:12.542499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2cafc7b5ce4dd5a727f72843198876'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:12.551036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2cafc7b5ce4dd5a727f72843198876'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:12.552291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2cafc7b5ce4dd5a727f72843198876'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:12.583913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 987/1970] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:27:12 2022] DELETE /identity/v3/users/7f2cafc7b5ce4dd5a727f72843198876 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:12.588621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a15b-ce32-4d07-8e7f-bc1e833f296d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:12.589175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a15b-ce32-4d07-8e7f-bc1e833f296d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:12.589280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a15b-ce32-4d07-8e7f-bc1e833f296d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:12.612131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2d93a15b-ce32-4d07-8e7f-bc1e833f296d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:12.613457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2d93a15b-ce32-4d07-8e7f-bc1e833f296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:12.682054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 984/1971] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:27:12.691898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:12.693393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:12.760126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02df1d2-92e9-4d45-aa19-46a58e1f452e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:12.763796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41938706-0bf3-4395-b6ce-329add35cfa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:12.764517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938706-0bf3-4395-b6ce-329add35cfa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:12.764794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938706-0bf3-4395-b6ce-329add35cfa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:12.765061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938706-0bf3-4395-b6ce-329add35cfa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:12.775198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41938706-0bf3-4395-b6ce-329add35cfa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:12.776824 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41938706-0bf3-4395-b6ce-329add35cfa7 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:12.787302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 988/1972] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:27:13.178369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93a15b-ce32-4d07-8e7f-bc1e833f296d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:13.179977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2d93a15b-ce32-4d07-8e7f-bc1e833f296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:13.214474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93a15b-ce32-4d07-8e7f-bc1e833f296d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:13.234201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:13.234890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.235161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.235420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] PATH_INFO: `/v3/projects/0c02fbce467e4e86826cdab9719f9ef5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.237663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c02fbce467e4e86826cdab9719f9ef5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:13.239814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:13.351684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c02fbce467e4e86826cdab9719f9ef5', 'request_id': 'req-407d2438-7278-4c9e-b5c0-23a05fba9f99'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:13.376205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c02fbce467e4e86826cdab9719f9ef5', 'request_id': 'req-407d2438-7278-4c9e-b5c0-23a05fba9f99'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:13.379704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:13.390228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c02fbce467e4e86826cdab9719f9ef5', 'request_id': 'req-407d2438-7278-4c9e-b5c0-23a05fba9f99'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:13.414760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] The token cache is being invalidate because project 0c02fbce467e4e86826cdab9719f9ef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:27:13.415210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:13.415763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-407d2438-7278-4c9e-b5c0-23a05fba9f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:13.417566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 985/1973] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:27:13 2022] DELETE /identity/v3/projects/0c02fbce467e4e86826cdab9719f9ef5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:13.422481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693cb9f-d0f8-469f-8c9a-77b25b56fa06 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.423024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693cb9f-d0f8-469f-8c9a-77b25b56fa06 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.423024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693cb9f-d0f8-469f-8c9a-77b25b56fa06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.432323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:13.446528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d693cb9f-d0f8-469f-8c9a-77b25b56fa06 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:13.447770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d693cb9f-d0f8-469f-8c9a-77b25b56fa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:13.509779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 989/1974] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:13.518907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fbdb2-1946-4133-a390-543001300e2f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.519183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fbdb2-1946-4133-a390-543001300e2f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.519450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fbdb2-1946-4133-a390-543001300e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.541114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e84fbdb2-1946-4133-a390-543001300e2f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:13.542233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e84fbdb2-1946-4133-a390-543001300e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:13.553450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 986/1975] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:13.554878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693cb9f-d0f8-469f-8c9a-77b25b56fa06 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:13.564976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693cb9f-d0f8-469f-8c9a-77b25b56fa06 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:13.570036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:13.570684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.570988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.571262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.571938 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:27:13.572240 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:13.573933 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:13.587008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 990/1976] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:27:13.656674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fbdb2-1946-4133-a390-543001300e2f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:13.667190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fbdb2-1946-4133-a390-543001300e2f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:13.673098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f2c3c-ea1c-48c6-9f40-25e756e50639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:13.673969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f2c3c-ea1c-48c6-9f40-25e756e50639 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.674497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f2c3c-ea1c-48c6-9f40-25e756e50639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.674958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f2c3c-ea1c-48c6-9f40-25e756e50639 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.675624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f2c3c-ea1c-48c6-9f40-25e756e50639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:13.683822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f2c3c-ea1c-48c6-9f40-25e756e50639 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:13.715676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 987/1977] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:27:13 2022] POST /identity/v3/projects => generated 395 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:27:13.720631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:13.732256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819d868-19fe-4458-9ff4-13e0d242971b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:13.820612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948ae13-e650-42c9-ba14-92197fe3d320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:13.821245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948ae13-e650-42c9-ba14-92197fe3d320 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.821538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948ae13-e650-42c9-ba14-92197fe3d320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.821897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948ae13-e650-42c9-ba14-92197fe3d320 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.826635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948ae13-e650-42c9-ba14-92197fe3d320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:13.828792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948ae13-e650-42c9-ba14-92197fe3d320 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:13.852003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 991/1978] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:27:13 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:27:13.856654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f2c3c-ea1c-48c6-9f40-25e756e50639 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:13.869325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f2c3c-ea1c-48c6-9f40-25e756e50639 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:13.874297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd491b3-f69f-4e34-b6e2-b45d95b8ce33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:13.875038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd491b3-f69f-4e34-b6e2-b45d95b8ce33 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.875413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd491b3-f69f-4e34-b6e2-b45d95b8ce33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.875865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd491b3-f69f-4e34-b6e2-b45d95b8ce33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.876598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd491b3-f69f-4e34-b6e2-b45d95b8ce33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:13.878253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd491b3-f69f-4e34-b6e2-b45d95b8ce33 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:13.893758 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 988/1979] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:27:13.897886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948ae13-e650-42c9-ba14-92197fe3d320 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:13.907993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948ae13-e650-42c9-ba14-92197fe3d320 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:13.914116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ebd86-480d-414d-8f70-4feaf31e07bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:13.914981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ebd86-480d-414d-8f70-4feaf31e07bd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.915619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ebd86-480d-414d-8f70-4feaf31e07bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.916012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ebd86-480d-414d-8f70-4feaf31e07bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.916880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ebd86-480d-414d-8f70-4feaf31e07bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:13.920744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ebd86-480d-414d-8f70-4feaf31e07bd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:13.933393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 992/1980] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:27:13.937784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd491b3-f69f-4e34-b6e2-b45d95b8ce33 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:13.948154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd491b3-f69f-4e34-b6e2-b45d95b8ce33 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:13.952756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:13.953416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:13.953692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:13.953978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] PATH_INFO: `/v3/projects/b107de0985844a079ffc422695d1c5eb/users/a8a6b3d875334b03ae02619dcf924b32/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:13.977927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b107de0985844a079ffc422695d1c5eb, user_id=a8a6b3d875334b03ae02619dcf924b32, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:13.981134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:13.995604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 989/1981] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:27:13 2022] PUT /identity/v3/projects/b107de0985844a079ffc422695d1c5eb/users/a8a6b3d875334b03ae02619dcf924b32/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:27:14.005604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512254c1-f0ba-41c4-a2ef-314aee17c2fe None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:14.005889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512254c1-f0ba-41c4-a2ef-314aee17c2fe None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:14.006259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512254c1-f0ba-41c4-a2ef-314aee17c2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:14.034762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-512254c1-f0ba-41c4-a2ef-314aee17c2fe None None] MFA Rules not processed for user `a8a6b3d875334b03ae02619dcf924b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:14.036012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-512254c1-f0ba-41c4-a2ef-314aee17c2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:14.112883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:14.122809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 993/1982] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:14.131343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:14.133148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:14.200062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fbc30-8dfd-406a-92ab-7fff60dac54a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:14.203688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11274841-7912-48e9-b0ff-794d9ee260b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:14.204291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11274841-7912-48e9-b0ff-794d9ee260b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:14.204562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11274841-7912-48e9-b0ff-794d9ee260b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:14.204826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11274841-7912-48e9-b0ff-794d9ee260b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:14.214445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11274841-7912-48e9-b0ff-794d9ee260b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:14.215863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11274841-7912-48e9-b0ff-794d9ee260b8 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:14.225840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 990/1983] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:14 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:27:24.220731 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=3,keystone:SELECT=57,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:40.342524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512254c1-f0ba-41c4-a2ef-314aee17c2fe None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:40.404343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512254c1-f0ba-41c4-a2ef-314aee17c2fe None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:40.408777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:40.409811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:40.410092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:40.410390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] PATH_INFO: `/v3/users/a8a6b3d875334b03ae02619dcf924b32` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:40.411515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a6b3d875334b03ae02619dcf924b32)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:40.413432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:40.457976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a6b3d875334b03ae02619dcf924b32'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:40.458635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a6b3d875334b03ae02619dcf924b32'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:40.470914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a6b3d875334b03ae02619dcf924b32'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:40.478417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a6b3d875334b03ae02619dcf924b32'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:40.479642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a6b3d875334b03ae02619dcf924b32'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:40.491834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 994/1984] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:27:40 2022] DELETE /identity/v3/users/a8a6b3d875334b03ae02619dcf924b32 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:40.496638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42085607-b491-4db6-8bdf-50b3d1b2f3d1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:40.497114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42085607-b491-4db6-8bdf-50b3d1b2f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:40.497235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42085607-b491-4db6-8bdf-50b3d1b2f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:40.516325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-42085607-b491-4db6-8bdf-50b3d1b2f3d1 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:40.517547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-42085607-b491-4db6-8bdf-50b3d1b2f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:40.579161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 991/1985] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:40.586224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:40.587696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:40.664258 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca87f7b-bc7c-40f4-9019-3247e9dd99b5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:40.668851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1628633-0220-4c7a-90f8-7d22520cf70f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:40.669564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628633-0220-4c7a-90f8-7d22520cf70f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:40.669849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628633-0220-4c7a-90f8-7d22520cf70f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:40.670124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628633-0220-4c7a-90f8-7d22520cf70f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:40.680659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1628633-0220-4c7a-90f8-7d22520cf70f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:40.682135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1628633-0220-4c7a-90f8-7d22520cf70f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:40.694593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 995/1986] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:27:41.055859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42085607-b491-4db6-8bdf-50b3d1b2f3d1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:41.058083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-42085607-b491-4db6-8bdf-50b3d1b2f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:41.086453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42085607-b491-4db6-8bdf-50b3d1b2f3d1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:41.091280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:41.091913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:41.092298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:41.092621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] PATH_INFO: `/v3/projects/b107de0985844a079ffc422695d1c5eb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:41.093859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=b107de0985844a079ffc422695d1c5eb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:41.095881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:41.149890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b107de0985844a079ffc422695d1c5eb', 'request_id': 'req-398825fd-f781-4db6-bf16-35a9d5e379ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:41.156369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b107de0985844a079ffc422695d1c5eb', 'request_id': 'req-398825fd-f781-4db6-bf16-35a9d5e379ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:41.161927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b107de0985844a079ffc422695d1c5eb', 'request_id': 'req-398825fd-f781-4db6-bf16-35a9d5e379ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:41.178940 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] The token cache is being invalidate because project b107de0985844a079ffc422695d1c5eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:27:41.179232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:41.179874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:41.181810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 992/1987] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:27:41 2022] DELETE /identity/v3/projects/b107de0985844a079ffc422695d1c5eb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:46.585872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1628633-0220-4c7a-90f8-7d22520cf70f service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:46.595118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1628633-0220-4c7a-90f8-7d22520cf70f service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:46.613847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:46.648953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31f1ad-0854-47ee-b1be-8cd897a17d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:46.649564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f1ad-0854-47ee-b1be-8cd897a17d95 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:46.649832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f1ad-0854-47ee-b1be-8cd897a17d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:46.650107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f1ad-0854-47ee-b1be-8cd897a17d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:46.652528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b31f1ad-0854-47ee-b1be-8cd897a17d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:46.718749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31f1ad-0854-47ee-b1be-8cd897a17d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:46.720232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31f1ad-0854-47ee-b1be-8cd897a17d95 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:46.731386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 996/1988] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:27:51.175619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:55.264824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:55.266391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:55.339025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398825fd-f781-4db6-bf16-35a9d5e379ab admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:55.343765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:55.344456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:55.344706 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:55.345059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] PATH_INFO: `/v3/users/f06f2e1909cc47adab776215c7feab88` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:55.346478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06f2e1909cc47adab776215c7feab88)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:55.348825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:55.415882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06f2e1909cc47adab776215c7feab88'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:55.417004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06f2e1909cc47adab776215c7feab88'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:55.425739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06f2e1909cc47adab776215c7feab88'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:55.440504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06f2e1909cc47adab776215c7feab88'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:55.441216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06f2e1909cc47adab776215c7feab88'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:55.455152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 993/1989] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:27:55 2022] DELETE /identity/v3/users/f06f2e1909cc47adab776215c7feab88 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:55.459551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2e7fd-5748-4dc2-a275-2056bf2d8fe5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:55.460035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2e7fd-5748-4dc2-a275-2056bf2d8fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:55.460118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2e7fd-5748-4dc2-a275-2056bf2d8fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:55.483741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-aab2e7fd-5748-4dc2-a275-2056bf2d8fe5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:55.485217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aab2e7fd-5748-4dc2-a275-2056bf2d8fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:55.552636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 997/1990] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:55.564000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:55.566072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:55.654874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e18b8-7f1e-41a7-84eb-db8564a0ce2d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:55.674722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71d87e-fb93-4eb5-8fab-084aadf949d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:55.675554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71d87e-fb93-4eb5-8fab-084aadf949d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:55.675789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71d87e-fb93-4eb5-8fab-084aadf949d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:55.676150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71d87e-fb93-4eb5-8fab-084aadf949d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:55.688531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71d87e-fb93-4eb5-8fab-084aadf949d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:55.690249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71d87e-fb93-4eb5-8fab-084aadf949d5 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:55.702316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 994/1991] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:27:56.138118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2e7fd-5748-4dc2-a275-2056bf2d8fe5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:56.139541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aab2e7fd-5748-4dc2-a275-2056bf2d8fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:56.154446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2e7fd-5748-4dc2-a275-2056bf2d8fe5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:56.159324 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:56.159951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:56.160289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:56.160562 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] PATH_INFO: `/v3/projects/f8212d6289834a9899dadb01d2f10e28` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:56.161669 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8212d6289834a9899dadb01d2f10e28)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:56.163618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:56.218744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8212d6289834a9899dadb01d2f10e28', 'request_id': 'req-d030d9b8-45d0-44c8-aa58-c99e875d7106'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.225270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8212d6289834a9899dadb01d2f10e28', 'request_id': 'req-d030d9b8-45d0-44c8-aa58-c99e875d7106'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.233758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8212d6289834a9899dadb01d2f10e28', 'request_id': 'req-d030d9b8-45d0-44c8-aa58-c99e875d7106'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.240209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:56.251511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] The token cache is being invalidate because project f8212d6289834a9899dadb01d2f10e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:27:56.252576 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.253687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.256620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 998/1992] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:27:56 2022] DELETE /identity/v3/projects/f8212d6289834a9899dadb01d2f10e28 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:56.261148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71d87e-fb93-4eb5-8fab-084aadf949d5 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:56.272712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71d87e-fb93-4eb5-8fab-084aadf949d5 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:56.291802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:56.335979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:56.336954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:56.336954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:56.337451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] PATH_INFO: `/v3/users/8418a1d5fb464d4d8653a174ab71de3d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:56.339114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8418a1d5fb464d4d8653a174ab71de3d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:56.341057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:56.389202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8418a1d5fb464d4d8653a174ab71de3d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.390032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8418a1d5fb464d4d8653a174ab71de3d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.398149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8418a1d5fb464d4d8653a174ab71de3d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.411243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8418a1d5fb464d4d8653a174ab71de3d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.411900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3a190d7c-cf07-4462-8b19-9853c37ae4a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8418a1d5fb464d4d8653a174ab71de3d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.424097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 995/1993] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:27:56 2022] DELETE /identity/v3/users/8418a1d5fb464d4d8653a174ab71de3d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:27:56.708579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:56.710069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:56.726107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030d9b8-45d0-44c8-aa58-c99e875d7106 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:56.794508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:56.795104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:56.795582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:56.795921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] PATH_INFO: `/v3/projects/bd50088ea246418297404cf41b59e30a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:56.797062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd50088ea246418297404cf41b59e30a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:56.799008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:56.860595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd50088ea246418297404cf41b59e30a', 'request_id': 'req-3b14b32d-6942-41f5-9d78-e219f3c70f1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.868218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd50088ea246418297404cf41b59e30a', 'request_id': 'req-3b14b32d-6942-41f5-9d78-e219f3c70f1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.878580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd50088ea246418297404cf41b59e30a', 'request_id': 'req-3b14b32d-6942-41f5-9d78-e219f3c70f1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.902474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] The token cache is being invalidate because project bd50088ea246418297404cf41b59e30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:27:56.902771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.903821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b14b32d-6942-41f5-9d78-e219f3c70f1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:56.909022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 999/1994] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:27:56 2022] DELETE /identity/v3/projects/bd50088ea246418297404cf41b59e30a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:56.910929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13142b-203b-42ed-a108-ad1334003ae7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:56.911216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13142b-203b-42ed-a108-ad1334003ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:56.911517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13142b-203b-42ed-a108-ad1334003ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:56.932739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5b13142b-203b-42ed-a108-ad1334003ae7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:56.933940 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b13142b-203b-42ed-a108-ad1334003ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:57.016501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 996/1995] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:57.026089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc5e5-3c5f-4f42-83f5-37df38679a0e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.026492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc5e5-3c5f-4f42-83f5-37df38679a0e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.026806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc5e5-3c5f-4f42-83f5-37df38679a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.048341 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-eafdc5e5-3c5f-4f42-83f5-37df38679a0e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:57.051252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eafdc5e5-3c5f-4f42-83f5-37df38679a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:57.062024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1000/1996] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:57.063306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13142b-203b-42ed-a108-ad1334003ae7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.073981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13142b-203b-42ed-a108-ad1334003ae7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.082009 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.082650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.082987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.083378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.084037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:27:57.084397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.086095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.101018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 997/1997] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:27:57.106277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc5e5-3c5f-4f42-83f5-37df38679a0e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.115833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc5e5-3c5f-4f42-83f5-37df38679a0e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.120648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd960a-e90b-4c39-ac2b-44890e6875b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.121253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd960a-e90b-4c39-ac2b-44890e6875b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.121530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd960a-e90b-4c39-ac2b-44890e6875b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.121798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd960a-e90b-4c39-ac2b-44890e6875b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.122463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd960a-e90b-4c39-ac2b-44890e6875b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.124371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd960a-e90b-4c39-ac2b-44890e6875b9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.165239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1001/1998] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:27:57 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:27:57.169953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.184601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615445a-1ce5-4183-b3bd-0a6f7cdc50b5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.219444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:57.257058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731695e-60d9-428c-8966-38be0f29b710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.257674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731695e-60d9-428c-8966-38be0f29b710 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.257945 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731695e-60d9-428c-8966-38be0f29b710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.258240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731695e-60d9-428c-8966-38be0f29b710 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.258933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731695e-60d9-428c-8966-38be0f29b710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.261635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731695e-60d9-428c-8966-38be0f29b710 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.289885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 998/1999] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:27:57 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:27:57.291298 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd960a-e90b-4c39-ac2b-44890e6875b9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.303295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd960a-e90b-4c39-ac2b-44890e6875b9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.308751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6ef9-ff66-4697-8adc-fffb842e0438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.309421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6ef9-ff66-4697-8adc-fffb842e0438 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.309693 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6ef9-ff66-4697-8adc-fffb842e0438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.309962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6ef9-ff66-4697-8adc-fffb842e0438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.310648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a6ef9-ff66-4697-8adc-fffb842e0438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.312230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a6ef9-ff66-4697-8adc-fffb842e0438 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.323478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1002/2000] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:27:57.328036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731695e-60d9-428c-8966-38be0f29b710 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.338850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731695e-60d9-428c-8966-38be0f29b710 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.343495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ade86f-d819-4010-92eb-045c40876af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.344098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade86f-d819-4010-92eb-045c40876af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.344377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade86f-d819-4010-92eb-045c40876af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.344647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade86f-d819-4010-92eb-045c40876af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.345310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ade86f-d819-4010-92eb-045c40876af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.346924 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ade86f-d819-4010-92eb-045c40876af5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.358638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 999/2001] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:27:57.364151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6ef9-ff66-4697-8adc-fffb842e0438 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.375593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6ef9-ff66-4697-8adc-fffb842e0438 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.380169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1d30b-856f-4c14-9ea5-9efe58ce2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.380813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1d30b-856f-4c14-9ea5-9efe58ce2d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.381086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1d30b-856f-4c14-9ea5-9efe58ce2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.381387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1d30b-856f-4c14-9ea5-9efe58ce2d15 admin admin] PATH_INFO: `/v3/projects/26165d67d72b4ce6bcecd15b74246aad/users/0acb9d8984c440c68bf2555e56bedde7/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.398407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1d30b-856f-4c14-9ea5-9efe58ce2d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26165d67d72b4ce6bcecd15b74246aad, user_id=0acb9d8984c440c68bf2555e56bedde7, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.401184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1d30b-856f-4c14-9ea5-9efe58ce2d15 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.415382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1003/2002] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:27:57 2022] PUT /identity/v3/projects/26165d67d72b4ce6bcecd15b74246aad/users/0acb9d8984c440c68bf2555e56bedde7/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:57.426822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cfa0-1c8a-48c7-b4e9-cd2e4c740ef3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.427180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cfa0-1c8a-48c7-b4e9-cd2e4c740ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.427522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cfa0-1c8a-48c7-b4e9-cd2e4c740ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.459144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-db03cfa0-1c8a-48c7-b4e9-cd2e4c740ef3 None None] MFA Rules not processed for user `0acb9d8984c440c68bf2555e56bedde7`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:57.460039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-db03cfa0-1c8a-48c7-b4e9-cd2e4c740ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:57.537563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1000/2003] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:57.542508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1d30b-856f-4c14-9ea5-9efe58ce2d15 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.571734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:57.622849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1d30b-856f-4c14-9ea5-9efe58ce2d15 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.629230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03498-8d6d-44a3-84b7-65943df395bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.629870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03498-8d6d-44a3-84b7-65943df395bd admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.630151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03498-8d6d-44a3-84b7-65943df395bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.630480 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03498-8d6d-44a3-84b7-65943df395bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.631193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e03498-8d6d-44a3-84b7-65943df395bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.633941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e03498-8d6d-44a3-84b7-65943df395bd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.674740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1004/2004] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:27:57 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:27:57.678683 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03cfa0-1c8a-48c7-b4e9-cd2e4c740ef3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.692619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03cfa0-1c8a-48c7-b4e9-cd2e4c740ef3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.784499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce417f4-768f-48b5-b04d-b3188c85d2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.785154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce417f4-768f-48b5-b04d-b3188c85d2aa admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.785634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce417f4-768f-48b5-b04d-b3188c85d2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.785903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce417f4-768f-48b5-b04d-b3188c85d2aa admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.787115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce417f4-768f-48b5-b04d-b3188c85d2aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.789321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce417f4-768f-48b5-b04d-b3188c85d2aa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.822272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1001/2005] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:27:57 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:27:57.827423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03498-8d6d-44a3-84b7-65943df395bd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.844803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03498-8d6d-44a3-84b7-65943df395bd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.850812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d2431-2591-4ff1-890b-00dc4ab5dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.852067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d2431-2591-4ff1-890b-00dc4ab5dc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.852343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d2431-2591-4ff1-890b-00dc4ab5dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.852788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d2431-2591-4ff1-890b-00dc4ab5dc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.854442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d2431-2591-4ff1-890b-00dc4ab5dc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.857064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d2431-2591-4ff1-890b-00dc4ab5dc04 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.874625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1005/2006] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:27:57.877355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce417f4-768f-48b5-b04d-b3188c85d2aa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.894563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce417f4-768f-48b5-b04d-b3188c85d2aa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:57.901123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:57.901899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:57.902180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:57.902647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] PATH_INFO: `/v3/projects/26888ab3a1014bb9976ab6dec006a8e7/users/21dcffdf676343fcb92b3297a9d74e9d/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:57.921803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=26888ab3a1014bb9976ab6dec006a8e7, user_id=21dcffdf676343fcb92b3297a9d74e9d, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:57.924569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:57.942606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1002/2007] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:27:57 2022] PUT /identity/v3/projects/26888ab3a1014bb9976ab6dec006a8e7/users/21dcffdf676343fcb92b3297a9d74e9d/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:27:57.946915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d2431-2591-4ff1-890b-00dc4ab5dc04 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:57.959477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d2431-2591-4ff1-890b-00dc4ab5dc04 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:58.024441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc1917-b2db-4a09-99b5-55fba4e58faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:58.025053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1917-b2db-4a09-99b5-55fba4e58faf admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.025366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1917-b2db-4a09-99b5-55fba4e58faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.025630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1917-b2db-4a09-99b5-55fba4e58faf admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.026302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc1917-b2db-4a09-99b5-55fba4e58faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:58.027921 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc1917-b2db-4a09-99b5-55fba4e58faf admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:58.037874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1006/2008] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:27:58.042210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc1917-b2db-4a09-99b5-55fba4e58faf admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:58.042663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:58.044815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.052481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc1917-b2db-4a09-99b5-55fba4e58faf admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:58.052992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:58.059319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90444a5c-e4db-417c-9d6b-35895929b93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:58.059909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90444a5c-e4db-417c-9d6b-35895929b93b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.060188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90444a5c-e4db-417c-9d6b-35895929b93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.060353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41282e18-4f0c-46eb-8229-974ca8a601cf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:58.060469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90444a5c-e4db-417c-9d6b-35895929b93b admin admin] PATH_INFO: `/v3/projects/26888ab3a1014bb9976ab6dec006a8e7/users/21dcffdf676343fcb92b3297a9d74e9d/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.062441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90444a5c-e4db-417c-9d6b-35895929b93b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26888ab3a1014bb9976ab6dec006a8e7, user_id=21dcffdf676343fcb92b3297a9d74e9d, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:58.065137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90444a5c-e4db-417c-9d6b-35895929b93b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:58.065834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:58.066515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.066792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.067061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] PATH_INFO: `/v3/users/6985b102dfd84ed0aee49b3d420007ca` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.070452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6985b102dfd84ed0aee49b3d420007ca)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:58.070715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:58.076301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1007/2009] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:27:58 2022] PUT /identity/v3/projects/26888ab3a1014bb9976ab6dec006a8e7/users/21dcffdf676343fcb92b3297a9d74e9d/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:27:58.086510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6130-7bb1-40c8-8089-59396f40336f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.086791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6130-7bb1-40c8-8089-59396f40336f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.087056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6130-7bb1-40c8-8089-59396f40336f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.118867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-827e6130-7bb1-40c8-8089-59396f40336f None None] MFA Rules not processed for user `21dcffdf676343fcb92b3297a9d74e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:58.120535 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-827e6130-7bb1-40c8-8089-59396f40336f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.130413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6985b102dfd84ed0aee49b3d420007ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.131242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6985b102dfd84ed0aee49b3d420007ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.139276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6985b102dfd84ed0aee49b3d420007ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.152960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6985b102dfd84ed0aee49b3d420007ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.153696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fc380da9-a596-477a-98a9-9c33667fef39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6985b102dfd84ed0aee49b3d420007ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.167391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1003/2010] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:27:58 2022] DELETE /identity/v3/users/6985b102dfd84ed0aee49b3d420007ca => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:58.171797 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090adf-b894-49d5-8a6c-a3ef293b62d1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.172071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090adf-b894-49d5-8a6c-a3ef293b62d1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.172340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090adf-b894-49d5-8a6c-a3ef293b62d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.192230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-50090adf-b894-49d5-8a6c-a3ef293b62d1 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:58.192642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1008/2011] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:27:58.193508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-50090adf-b894-49d5-8a6c-a3ef293b62d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.204129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e6130-7bb1-40c8-8089-59396f40336f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:58.205583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-827e6130-7bb1-40c8-8089-59396f40336f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.211863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:58.264170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1004/2012] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:58.270703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50090adf-b894-49d5-8a6c-a3ef293b62d1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:58.272090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-50090adf-b894-49d5-8a6c-a3ef293b62d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.283251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e6130-7bb1-40c8-8089-59396f40336f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:58.339824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50090adf-b894-49d5-8a6c-a3ef293b62d1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:58.341936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31702716-46b0-4190-93be-0a13fa8fb02f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:58.342690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702716-46b0-4190-93be-0a13fa8fb02f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.343033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702716-46b0-4190-93be-0a13fa8fb02f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.343392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702716-46b0-4190-93be-0a13fa8fb02f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.343985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:58.344639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.344977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.345313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.357730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:58.359435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:58.370031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1005/2013] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:27:58.415529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31702716-46b0-4190-93be-0a13fa8fb02f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:58.417798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31702716-46b0-4190-93be-0a13fa8fb02f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:58.428804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1009/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:27:58.459563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:58.461123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.475017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0368f88-5dd9-401b-a94a-c1562f3ed886 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:58.480500 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:58.481058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.481383 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.481633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] PATH_INFO: `/v3/users/0acb9d8984c440c68bf2555e56bedde7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.482829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0acb9d8984c440c68bf2555e56bedde7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:58.484790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:58.499651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:58.548214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0acb9d8984c440c68bf2555e56bedde7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.548883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0acb9d8984c440c68bf2555e56bedde7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.557187 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0acb9d8984c440c68bf2555e56bedde7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.571252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0acb9d8984c440c68bf2555e56bedde7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.572148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0acb9d8984c440c68bf2555e56bedde7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.586068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1006/2015] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:27:58 2022] DELETE /identity/v3/users/0acb9d8984c440c68bf2555e56bedde7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:58.591733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89f7fb-eaf5-4eeb-b43b-d51dfa6665c4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.592091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89f7fb-eaf5-4eeb-b43b-d51dfa6665c4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.592473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89f7fb-eaf5-4eeb-b43b-d51dfa6665c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.612685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-aa89f7fb-eaf5-4eeb-b43b-d51dfa6665c4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:58.613918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aa89f7fb-eaf5-4eeb-b43b-d51dfa6665c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.662085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:58.664986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.684632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1010/2016] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:58.694780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89f7fb-eaf5-4eeb-b43b-d51dfa6665c4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:58.696169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aa89f7fb-eaf5-4eeb-b43b-d51dfa6665c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.711422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:58.747772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63635-dc8b-4aa5-84dc-2bd402c8e81c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:58.752634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:58.753258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.753533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.753801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] PATH_INFO: `/v3/projects/c45638677b3540e281a8c5771d1dca24` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.755008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c45638677b3540e281a8c5771d1dca24)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:58.757002 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:58.759092 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89f7fb-eaf5-4eeb-b43b-d51dfa6665c4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:58.764882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58726a47-6f3f-4226-b4ff-8457573a4706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:58.765801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58726a47-6f3f-4226-b4ff-8457573a4706 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.766579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58726a47-6f3f-4226-b4ff-8457573a4706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.767021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58726a47-6f3f-4226-b4ff-8457573a4706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.778013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58726a47-6f3f-4226-b4ff-8457573a4706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:58.779545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58726a47-6f3f-4226-b4ff-8457573a4706 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:58.791113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1011/2017] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:27:58.808834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45638677b3540e281a8c5771d1dca24', 'request_id': 'req-4b8debad-3245-4769-b21b-94a2609298b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.819217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45638677b3540e281a8c5771d1dca24', 'request_id': 'req-4b8debad-3245-4769-b21b-94a2609298b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.826087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45638677b3540e281a8c5771d1dca24', 'request_id': 'req-4b8debad-3245-4769-b21b-94a2609298b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.840253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] The token cache is being invalidate because project c45638677b3540e281a8c5771d1dca24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:27:58.840671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.841385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b8debad-3245-4769-b21b-94a2609298b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:58.843039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1007/2018] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:27:58 2022] DELETE /identity/v3/projects/c45638677b3540e281a8c5771d1dca24 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:58.849960 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d923a-7671-4214-bcca-2115b337be2a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.850475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d923a-7671-4214-bcca-2115b337be2a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.850602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d923a-7671-4214-bcca-2115b337be2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.871759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d49d923a-7671-4214-bcca-2115b337be2a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:58.872966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d49d923a-7671-4214-bcca-2115b337be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.955335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1012/2019] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:27:58.960192 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a27e6e-2d51-4fc5-be6d-46ce4ae19fcf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:58.961022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a27e6e-2d51-4fc5-be6d-46ce4ae19fcf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:58.961286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a27e6e-2d51-4fc5-be6d-46ce4ae19fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:58.986764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d4a27e6e-2d51-4fc5-be6d-46ce4ae19fcf None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:58.987640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d4a27e6e-2d51-4fc5-be6d-46ce4ae19fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:58.998059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1008/2020] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:27:59.002363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d923a-7671-4214-bcca-2115b337be2a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.022460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d923a-7671-4214-bcca-2115b337be2a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.032666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.033435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.033789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.034162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.035208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:27:59.036456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.039574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.059510 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1013/2021] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:27:59.061034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a27e6e-2d51-4fc5-be6d-46ce4ae19fcf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.071718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a27e6e-2d51-4fc5-be6d-46ce4ae19fcf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.081231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.082234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.082901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.083985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.084746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.086714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.102692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:59.127343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1009/2022] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:27:59 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:27:59.131219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.138129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.142582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:59.145481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017552-8ee2-4c5d-80d2-6196a022d7a0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.228597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e5507-1e7d-4960-bf40-72f080e1310b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.233642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96852d3-0917-40fc-a7de-60f708cc8bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.234269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.234456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96852d3-0917-40fc-a7de-60f708cc8bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.234750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96852d3-0917-40fc-a7de-60f708cc8bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.235134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.235246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96852d3-0917-40fc-a7de-60f708cc8bd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.235585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.236037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] PATH_INFO: `/v3/projects/26165d67d72b4ce6bcecd15b74246aad` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.236458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96852d3-0917-40fc-a7de-60f708cc8bd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.237222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26165d67d72b4ce6bcecd15b74246aad)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.238648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96852d3-0917-40fc-a7de-60f708cc8bd5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.239364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.272380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1014/2023] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:27:59 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:27:59.273514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96852d3-0917-40fc-a7de-60f708cc8bd5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.285878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96852d3-0917-40fc-a7de-60f708cc8bd5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.293188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfda7f4-66e6-4559-b420-12f981fda7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.293896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda7f4-66e6-4559-b420-12f981fda7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.294436 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda7f4-66e6-4559-b420-12f981fda7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.294766 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda7f4-66e6-4559-b420-12f981fda7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.295418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfda7f4-66e6-4559-b420-12f981fda7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.297565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfda7f4-66e6-4559-b420-12f981fda7b5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.307171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26165d67d72b4ce6bcecd15b74246aad', 'request_id': 'req-31d83b2a-19c1-4834-83aa-94b61f2f7194'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.313893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1015/2024] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:27:59.314666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26165d67d72b4ce6bcecd15b74246aad', 'request_id': 'req-31d83b2a-19c1-4834-83aa-94b61f2f7194'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.317825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfda7f4-66e6-4559-b420-12f981fda7b5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.322454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26165d67d72b4ce6bcecd15b74246aad', 'request_id': 'req-31d83b2a-19c1-4834-83aa-94b61f2f7194'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.330455 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfda7f4-66e6-4559-b420-12f981fda7b5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.337186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] The token cache is being invalidate because project 26165d67d72b4ce6bcecd15b74246aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:27:59.337556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.339325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.341243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1010/2025] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:27:59 2022] DELETE /identity/v3/projects/26165d67d72b4ce6bcecd15b74246aad => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:59.345040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.346528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:59.350879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:59.397610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.398387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.398709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.398996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.399678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.402526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.417585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1016/2026] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:27:59.419031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.423205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:59.431000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d83b2a-19c1-4834-83aa-94b61f2f7194 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.436110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.436780 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.437052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.437331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] PATH_INFO: `/v3/users/21dcffdf676343fcb92b3297a9d74e9d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.438831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bcf0f-8ae8-449e-b137-1d119f66a40e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.438939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21dcffdf676343fcb92b3297a9d74e9d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.440814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.443577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f295e1-e737-4a46-a373-8f168f825cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.444238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f295e1-e737-4a46-a373-8f168f825cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.444515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f295e1-e737-4a46-a373-8f168f825cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.444785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f295e1-e737-4a46-a373-8f168f825cbc admin admin] PATH_INFO: `/v3/projects/8ca835fdc95849cd8db1882ebb693cc3/users/75018aeecd6843c0a75bc95e8757fd8a/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.462771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f295e1-e737-4a46-a373-8f168f825cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca835fdc95849cd8db1882ebb693cc3, user_id=75018aeecd6843c0a75bc95e8757fd8a, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.466086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f295e1-e737-4a46-a373-8f168f825cbc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.482218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1017/2027] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:27:59 2022] PUT /identity/v3/projects/8ca835fdc95849cd8db1882ebb693cc3/users/75018aeecd6843c0a75bc95e8757fd8a/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:27:59.487426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e7cd5-d1ba-4e2f-8216-e6e3d506a3c6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.487711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e7cd5-d1ba-4e2f-8216-e6e3d506a3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.488038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e7cd5-d1ba-4e2f-8216-e6e3d506a3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.494315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:27:59.512566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-159e7cd5-d1ba-4e2f-8216-e6e3d506a3c6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:27:59.513829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-159e7cd5-d1ba-4e2f-8216-e6e3d506a3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:59.530913 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21dcffdf676343fcb92b3297a9d74e9d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.531648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21dcffdf676343fcb92b3297a9d74e9d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.545351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21dcffdf676343fcb92b3297a9d74e9d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.568691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21dcffdf676343fcb92b3297a9d74e9d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.569435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21dcffdf676343fcb92b3297a9d74e9d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:27:59.586417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1011/2028] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:27:59 2022] DELETE /identity/v3/users/21dcffdf676343fcb92b3297a9d74e9d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:27:59.598856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1018/2029] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:27:59.612489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.613990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:59.695344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ea82f-7470-41b4-9715-f526b297a270 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.699453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af925cc-1b24-4b0a-933d-a65cff09a9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.700058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af925cc-1b24-4b0a-933d-a65cff09a9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.700338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af925cc-1b24-4b0a-933d-a65cff09a9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.700608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af925cc-1b24-4b0a-933d-a65cff09a9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.767029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af925cc-1b24-4b0a-933d-a65cff09a9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.769016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af925cc-1b24-4b0a-933d-a65cff09a9d1 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:27:59.779405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1012/2030] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:27:59.927674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e7cd5-d1ba-4e2f-8216-e6e3d506a3c6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:27:59.929211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-159e7cd5-d1ba-4e2f-8216-e6e3d506a3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:27:59.943399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e7cd5-d1ba-4e2f-8216-e6e3d506a3c6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:27:59.948787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:27:59.949422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:27:59.949746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:27:59.950069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] PATH_INFO: `/v3/projects/26888ab3a1014bb9976ab6dec006a8e7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:27:59.952346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=26888ab3a1014bb9976ab6dec006a8e7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:27:59.954409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.013417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26888ab3a1014bb9976ab6dec006a8e7', 'request_id': 'req-58ade54c-460b-4894-b460-5cf5f1c8adeb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:00.020034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26888ab3a1014bb9976ab6dec006a8e7', 'request_id': 'req-58ade54c-460b-4894-b460-5cf5f1c8adeb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:00.030086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26888ab3a1014bb9976ab6dec006a8e7', 'request_id': 'req-58ade54c-460b-4894-b460-5cf5f1c8adeb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:00.041673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] The token cache is being invalidate because project 26888ab3a1014bb9976ab6dec006a8e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:28:00.041958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:00.042639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-58ade54c-460b-4894-b460-5cf5f1c8adeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:00.044192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1019/2031] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:27:59 2022] DELETE /identity/v3/projects/26888ab3a1014bb9976ab6dec006a8e7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:00.049297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66536d8d-b818-4cf3-90c9-258a21ba83ae None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.049562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66536d8d-b818-4cf3-90c9-258a21ba83ae None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.049828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66536d8d-b818-4cf3-90c9-258a21ba83ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.069860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-66536d8d-b818-4cf3-90c9-258a21ba83ae None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:00.071488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-66536d8d-b818-4cf3-90c9-258a21ba83ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:00.124072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:00.150999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1013/2032] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:28:00.153864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463837c8-4d61-49ac-97eb-cb089de26d4f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.154164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463837c8-4d61-49ac-97eb-cb089de26d4f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.154469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463837c8-4d61-49ac-97eb-cb089de26d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.178747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-463837c8-4d61-49ac-97eb-cb089de26d4f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:00.179966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-463837c8-4d61-49ac-97eb-cb089de26d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:00.189724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1020/2033] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:00.193998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66536d8d-b818-4cf3-90c9-258a21ba83ae None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.204649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66536d8d-b818-4cf3-90c9-258a21ba83ae None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.209814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.210447 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.210725 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.210994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.211649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:28:00.211950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.214655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.232452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1014/2034] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:28:00.237079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463837c8-4d61-49ac-97eb-cb089de26d4f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.247162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463837c8-4d61-49ac-97eb-cb089de26d4f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.252516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9c373-e660-48de-8412-ead05ac1e4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.253258 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9c373-e660-48de-8412-ead05ac1e4ff admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.253539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9c373-e660-48de-8412-ead05ac1e4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.253805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9c373-e660-48de-8412-ead05ac1e4ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.254494 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9c373-e660-48de-8412-ead05ac1e4ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.256409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9c373-e660-48de-8412-ead05ac1e4ff admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.262877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:00.297539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1021/2035] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:28:00 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:28:00.298792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.309303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9996981-3566-4616-9dbd-6d903790520c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.376450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0fa5d-53ec-4d2b-95d0-08a501dc115b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.377099 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0fa5d-53ec-4d2b-95d0-08a501dc115b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.377270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0fa5d-53ec-4d2b-95d0-08a501dc115b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.377540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0fa5d-53ec-4d2b-95d0-08a501dc115b admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.378220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0fa5d-53ec-4d2b-95d0-08a501dc115b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.380166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0fa5d-53ec-4d2b-95d0-08a501dc115b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.402117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1015/2036] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:28:00 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:28:00.406082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9c373-e660-48de-8412-ead05ac1e4ff admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.417235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9c373-e660-48de-8412-ead05ac1e4ff admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.422955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3ac90-61f0-4a8a-bb18-f41f9c9f5842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.423580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3ac90-61f0-4a8a-bb18-f41f9c9f5842 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.423858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3ac90-61f0-4a8a-bb18-f41f9c9f5842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.424131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3ac90-61f0-4a8a-bb18-f41f9c9f5842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.424844 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3ac90-61f0-4a8a-bb18-f41f9c9f5842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.426663 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3ac90-61f0-4a8a-bb18-f41f9c9f5842 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.439860 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1022/2037] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:28:00.443909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0fa5d-53ec-4d2b-95d0-08a501dc115b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.459395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0fa5d-53ec-4d2b-95d0-08a501dc115b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.464384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93008aab-55b4-4e93-805d-d1c47d46380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.465020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93008aab-55b4-4e93-805d-d1c47d46380b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.465297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93008aab-55b4-4e93-805d-d1c47d46380b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.465568 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93008aab-55b4-4e93-805d-d1c47d46380b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.466212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93008aab-55b4-4e93-805d-d1c47d46380b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.467853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93008aab-55b4-4e93-805d-d1c47d46380b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.485371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1016/2038] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:28:00.485967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3ac90-61f0-4a8a-bb18-f41f9c9f5842 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.498364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3ac90-61f0-4a8a-bb18-f41f9c9f5842 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.503679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec5e17-9850-483c-ba2b-76841982dfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.504529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec5e17-9850-483c-ba2b-76841982dfce admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.504804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec5e17-9850-483c-ba2b-76841982dfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.505076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec5e17-9850-483c-ba2b-76841982dfce admin admin] PATH_INFO: `/v3/projects/5b45fcffcc1b4baeb28c0fa56385e2d0/users/55d3ef4c20cc43f4976dd4e478c2aedb/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.544330 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec5e17-9850-483c-ba2b-76841982dfce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b45fcffcc1b4baeb28c0fa56385e2d0, user_id=55d3ef4c20cc43f4976dd4e478c2aedb, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.547070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec5e17-9850-483c-ba2b-76841982dfce admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.563883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1023/2039] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:28:00 2022] PUT /identity/v3/projects/5b45fcffcc1b4baeb28c0fa56385e2d0/users/55d3ef4c20cc43f4976dd4e478c2aedb/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:28:00.574784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b483de7-7d44-4acd-98c5-cb3f051608f7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.574967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b483de7-7d44-4acd-98c5-cb3f051608f7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.575221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b483de7-7d44-4acd-98c5-cb3f051608f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.609748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4b483de7-7d44-4acd-98c5-cb3f051608f7 None None] MFA Rules not processed for user `55d3ef4c20cc43f4976dd4e478c2aedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:00.611043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b483de7-7d44-4acd-98c5-cb3f051608f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:00.695984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1017/2040] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:00.700115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec5e17-9850-483c-ba2b-76841982dfce admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.795615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec5e17-9850-483c-ba2b-76841982dfce admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.801569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59030de-3ac9-4412-9144-003134440b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.801913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59030de-3ac9-4412-9144-003134440b94 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.802185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59030de-3ac9-4412-9144-003134440b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.802805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59030de-3ac9-4412-9144-003134440b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.803131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59030de-3ac9-4412-9144-003134440b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.805193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59030de-3ac9-4412-9144-003134440b94 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.846081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1024/2041] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:28:00 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:28:00.847323 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b483de7-7d44-4acd-98c5-cb3f051608f7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.860210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b483de7-7d44-4acd-98c5-cb3f051608f7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.911892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:00.932446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb22755-fe51-4635-a988-7841cb3eba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.933168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb22755-fe51-4635-a988-7841cb3eba93 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.934068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb22755-fe51-4635-a988-7841cb3eba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.934068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb22755-fe51-4635-a988-7841cb3eba93 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.934620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb22755-fe51-4635-a988-7841cb3eba93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.936740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb22755-fe51-4635-a988-7841cb3eba93 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:00.965738 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1018/2042] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:28:00 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:28:00.970419 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59030de-3ac9-4412-9144-003134440b94 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:00.984272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59030de-3ac9-4412-9144-003134440b94 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:00.990515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ed74-6f31-48ec-a2c7-7fd16cdbeccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:00.991577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ed74-6f31-48ec-a2c7-7fd16cdbeccc admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:00.991919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ed74-6f31-48ec-a2c7-7fd16cdbeccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:00.993615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ed74-6f31-48ec-a2c7-7fd16cdbeccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:00.993615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ed74-6f31-48ec-a2c7-7fd16cdbeccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:00.995571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ed74-6f31-48ec-a2c7-7fd16cdbeccc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:01.007774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1025/2043] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:28:01.013560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb22755-fe51-4635-a988-7841cb3eba93 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:01.030063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb22755-fe51-4635-a988-7841cb3eba93 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:01.035961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a948-9158-4851-a6e4-8f1f0c06ab2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:01.038482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a948-9158-4851-a6e4-8f1f0c06ab2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:01.038779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a948-9158-4851-a6e4-8f1f0c06ab2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:01.039039 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a948-9158-4851-a6e4-8f1f0c06ab2e admin admin] PATH_INFO: `/v3/projects/7af8b09212634a04aa1893e372fcc4a6/users/be99368c8eb04a1891399dcb4bdba2c5/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:01.063021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51a948-9158-4851-a6e4-8f1f0c06ab2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af8b09212634a04aa1893e372fcc4a6, user_id=be99368c8eb04a1891399dcb4bdba2c5, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:01.067868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51a948-9158-4851-a6e4-8f1f0c06ab2e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:01.093121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1019/2044] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:28:01 2022] PUT /identity/v3/projects/7af8b09212634a04aa1893e372fcc4a6/users/be99368c8eb04a1891399dcb4bdba2c5/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:28:01.097941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ed74-6f31-48ec-a2c7-7fd16cdbeccc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:01.123667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ed74-6f31-48ec-a2c7-7fd16cdbeccc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:01.264515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce3b5-d14d-45b8-99c7-84cefe9f1d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:01.265146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce3b5-d14d-45b8-99c7-84cefe9f1d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:01.265425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce3b5-d14d-45b8-99c7-84cefe9f1d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:01.265701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce3b5-d14d-45b8-99c7-84cefe9f1d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:01.266401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce3b5-d14d-45b8-99c7-84cefe9f1d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:01.268199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce3b5-d14d-45b8-99c7-84cefe9f1d4b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:01.293330 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1026/2045] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:28:01.297619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a948-9158-4851-a6e4-8f1f0c06ab2e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:01.314406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a948-9158-4851-a6e4-8f1f0c06ab2e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:01.323471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:01.324167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:01.324451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:01.324777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] PATH_INFO: `/v3/projects/7af8b09212634a04aa1893e372fcc4a6/users/be99368c8eb04a1891399dcb4bdba2c5/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:01.329775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af8b09212634a04aa1893e372fcc4a6, user_id=be99368c8eb04a1891399dcb4bdba2c5, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:01.335736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:01.365892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1020/2046] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:28:01 2022] PUT /identity/v3/projects/7af8b09212634a04aa1893e372fcc4a6/users/be99368c8eb04a1891399dcb4bdba2c5/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:01.376741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fcbed-b8d6-4ad2-a6de-f33c35edd1b1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:01.377087 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fcbed-b8d6-4ad2-a6de-f33c35edd1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:01.377283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fcbed-b8d6-4ad2-a6de-f33c35edd1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:01.443806 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1c5fcbed-b8d6-4ad2-a6de-f33c35edd1b1 None None] MFA Rules not processed for user `be99368c8eb04a1891399dcb4bdba2c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:01.445446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1c5fcbed-b8d6-4ad2-a6de-f33c35edd1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:01.463752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:01.549314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1027/2047] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:01.564945 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:01.567691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:01.706870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318bfed-8440-47c9-b9a5-d8fc218075fa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:01.714600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:01.715648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:01.715921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:01.716186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:01.812561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:01.818040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:01.837400 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1021/2048] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:05.027385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642a3c6-aa23-479f-804a-774f3458fb4e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:05.028639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642a3c6-aa23-479f-804a-774f3458fb4e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:05.029176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642a3c6-aa23-479f-804a-774f3458fb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:05.083575 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5642a3c6-aa23-479f-804a-774f3458fb4e None None] MFA Rules not processed for user `75018aeecd6843c0a75bc95e8757fd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:05.086946 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5642a3c6-aa23-479f-804a-774f3458fb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:05.207484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1028/2049] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:05.212429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:05.215720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:05.290315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:05.307042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b92b1-13da-474b-a87a-1178440b4318 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:05.312239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89668c-348a-41a1-a87e-61537c6d17cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:05.312936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89668c-348a-41a1-a87e-61537c6d17cf admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:05.313275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89668c-348a-41a1-a87e-61537c6d17cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:05.313671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89668c-348a-41a1-a87e-61537c6d17cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:05.314423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89668c-348a-41a1-a87e-61537c6d17cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:05.316878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89668c-348a-41a1-a87e-61537c6d17cf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:05.355702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1022/2050] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:28:05 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:28:05.359374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642a3c6-aa23-479f-804a-774f3458fb4e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:05.377122 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642a3c6-aa23-479f-804a-774f3458fb4e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:05.472338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba1c05-798e-4d04-a16e-9026b77665cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:05.473159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1c05-798e-4d04-a16e-9026b77665cb admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:05.473685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1c05-798e-4d04-a16e-9026b77665cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:05.473826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1c05-798e-4d04-a16e-9026b77665cb admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:05.474611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba1c05-798e-4d04-a16e-9026b77665cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:05.476539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba1c05-798e-4d04-a16e-9026b77665cb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:05.505087 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1029/2051] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:28:05 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:28:05.509160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89668c-348a-41a1-a87e-61537c6d17cf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:05.523575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89668c-348a-41a1-a87e-61537c6d17cf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:05.528454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d79f8e-0983-43a0-9ce5-fcad109322ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:05.529198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d79f8e-0983-43a0-9ce5-fcad109322ed admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:05.529476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d79f8e-0983-43a0-9ce5-fcad109322ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:05.529742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d79f8e-0983-43a0-9ce5-fcad109322ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:05.530417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d79f8e-0983-43a0-9ce5-fcad109322ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:05.532001 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d79f8e-0983-43a0-9ce5-fcad109322ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:05.544100 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1023/2052] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:28:05.548249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba1c05-798e-4d04-a16e-9026b77665cb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:05.559056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba1c05-798e-4d04-a16e-9026b77665cb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:05.564087 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d07cc-bac1-47d4-8561-120cf98c5771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:05.564738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d07cc-bac1-47d4-8561-120cf98c5771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:05.565019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d07cc-bac1-47d4-8561-120cf98c5771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:05.565291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d07cc-bac1-47d4-8561-120cf98c5771 admin admin] PATH_INFO: `/v3/projects/fdd2f2b7ddfd420c8f133852dbeea70c/users/e71ce5b5697f43f786e949a940f10c1e/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:05.581127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d07cc-bac1-47d4-8561-120cf98c5771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd2f2b7ddfd420c8f133852dbeea70c, user_id=e71ce5b5697f43f786e949a940f10c1e, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:05.583834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d07cc-bac1-47d4-8561-120cf98c5771 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:05.596949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1030/2053] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:28:05 2022] PUT /identity/v3/projects/fdd2f2b7ddfd420c8f133852dbeea70c/users/e71ce5b5697f43f786e949a940f10c1e/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:05.600938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d79f8e-0983-43a0-9ce5-fcad109322ed admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:05.611890 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d79f8e-0983-43a0-9ce5-fcad109322ed admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:05.668583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b4a0-2aa2-484a-bf9f-6961178b143f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:05.669182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b4a0-2aa2-484a-bf9f-6961178b143f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:05.669454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b4a0-2aa2-484a-bf9f-6961178b143f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:05.669741 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b4a0-2aa2-484a-bf9f-6961178b143f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:05.670395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70b4a0-2aa2-484a-bf9f-6961178b143f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:05.671959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70b4a0-2aa2-484a-bf9f-6961178b143f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:05.682060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1024/2054] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:28:05.686249 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d07cc-bac1-47d4-8561-120cf98c5771 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:05.691622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:05.697964 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d07cc-bac1-47d4-8561-120cf98c5771 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:05.702994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:05.703732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:05.703993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:05.704264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] PATH_INFO: `/v3/projects/fdd2f2b7ddfd420c8f133852dbeea70c/users/e71ce5b5697f43f786e949a940f10c1e/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:05.706168 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd2f2b7ddfd420c8f133852dbeea70c, user_id=e71ce5b5697f43f786e949a940f10c1e, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:05.708804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:05.718929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1031/2055] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:28:05 2022] PUT /identity/v3/projects/fdd2f2b7ddfd420c8f133852dbeea70c/users/e71ce5b5697f43f786e949a940f10c1e/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:06.626114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b4a0-2aa2-484a-bf9f-6961178b143f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:06.694604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b4a0-2aa2-484a-bf9f-6961178b143f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:06.793548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f602caf-0ed8-49b2-ae54-4b7b332cf1f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:06.794719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602caf-0ed8-49b2-ae54-4b7b332cf1f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:06.795151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602caf-0ed8-49b2-ae54-4b7b332cf1f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:06.795596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602caf-0ed8-49b2-ae54-4b7b332cf1f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:06.837273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:06.884040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f602caf-0ed8-49b2-ae54-4b7b332cf1f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:06.904008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f602caf-0ed8-49b2-ae54-4b7b332cf1f0 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:06.932094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1025/2056] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:09.341777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:09.344395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:09.488810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd0f01-5624-4c5d-83bd-95822c11641d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:09.499466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:09.500084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:09.501061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:09.501386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:09.503738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:09.625456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:09.627259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:09.638950 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1032/2057] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:11.042087 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08494-251d-49e6-a606-3c4dfd78a17b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:11.042636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08494-251d-49e6-a606-3c4dfd78a17b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:11.042717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08494-251d-49e6-a606-3c4dfd78a17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:11.075228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f7f08494-251d-49e6-a606-3c4dfd78a17b None None] MFA Rules not processed for user `e71ce5b5697f43f786e949a940f10c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:11.076442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f7f08494-251d-49e6-a606-3c4dfd78a17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:11.165898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1026/2058] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:11.182364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:11.184236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:11.318070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612ace-428a-4cbd-bc92-5cfc0f047213 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:11.328876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55513be2-da50-49fc-810d-6c7dcd4976d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:11.330886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55513be2-da50-49fc-810d-6c7dcd4976d1 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:11.333512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55513be2-da50-49fc-810d-6c7dcd4976d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:11.334440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55513be2-da50-49fc-810d-6c7dcd4976d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:11.381884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:11.408792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55513be2-da50-49fc-810d-6c7dcd4976d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:11.410492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55513be2-da50-49fc-810d-6c7dcd4976d1 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:11.421094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1033/2059] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:12.480580 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f08494-251d-49e6-a606-3c4dfd78a17b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:12.624792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f08494-251d-49e6-a606-3c4dfd78a17b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:12.636468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:12.637160 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:12.637432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:12.637700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] PATH_INFO: `/v3/users/55d3ef4c20cc43f4976dd4e478c2aedb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:12.642210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d3ef4c20cc43f4976dd4e478c2aedb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:12.647869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:12.733210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d3ef4c20cc43f4976dd4e478c2aedb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:12.734364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d3ef4c20cc43f4976dd4e478c2aedb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:12.747449 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d3ef4c20cc43f4976dd4e478c2aedb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:12.750655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:12.774535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d3ef4c20cc43f4976dd4e478c2aedb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:12.776175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d3ef4c20cc43f4976dd4e478c2aedb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:12.796859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1027/2060] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:28:12 2022] DELETE /identity/v3/users/55d3ef4c20cc43f4976dd4e478c2aedb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:12.799104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01b018-9d58-4dbb-ba71-3beceb235d27 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:12.799572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01b018-9d58-4dbb-ba71-3beceb235d27 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:12.799786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01b018-9d58-4dbb-ba71-3beceb235d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:12.842279 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-af01b018-9d58-4dbb-ba71-3beceb235d27 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:12.845237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-af01b018-9d58-4dbb-ba71-3beceb235d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:12.942753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1034/2061] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:12.952937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:12.954714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:13.068551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c60c3-3ff1-4ba7-aac4-354e4fb1482f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:13.077405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8eb4c-5c93-4e82-b9e6-ba28a6a1f7a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:13.078161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8eb4c-5c93-4e82-b9e6-ba28a6a1f7a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:13.078663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8eb4c-5c93-4e82-b9e6-ba28a6a1f7a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:13.079045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8eb4c-5c93-4e82-b9e6-ba28a6a1f7a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:13.092102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8eb4c-5c93-4e82-b9e6-ba28a6a1f7a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:13.093519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8eb4c-5c93-4e82-b9e6-ba28a6a1f7a7 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:13.107311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1028/2062] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:13.590230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01b018-9d58-4dbb-ba71-3beceb235d27 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:13.592003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-af01b018-9d58-4dbb-ba71-3beceb235d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:13.615280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01b018-9d58-4dbb-ba71-3beceb235d27 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:13.622099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:13.622962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:13.623421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:13.623792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] PATH_INFO: `/v3/projects/5b45fcffcc1b4baeb28c0fa56385e2d0` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:13.625105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b45fcffcc1b4baeb28c0fa56385e2d0)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:13.627369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:13.701862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b45fcffcc1b4baeb28c0fa56385e2d0', 'request_id': 'req-80137732-4892-4a78-a9bb-75e018ad935e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.709075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b45fcffcc1b4baeb28c0fa56385e2d0', 'request_id': 'req-80137732-4892-4a78-a9bb-75e018ad935e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.720845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b45fcffcc1b4baeb28c0fa56385e2d0', 'request_id': 'req-80137732-4892-4a78-a9bb-75e018ad935e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.750759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] The token cache is being invalidate because project 5b45fcffcc1b4baeb28c0fa56385e2d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:28:13.751057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.751705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-80137732-4892-4a78-a9bb-75e018ad935e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.754363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1035/2063] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:28:13 2022] DELETE /identity/v3/projects/5b45fcffcc1b4baeb28c0fa56385e2d0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:13.758255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8eb4c-5c93-4e82-b9e6-ba28a6a1f7a7 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:13.770094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8eb4c-5c93-4e82-b9e6-ba28a6a1f7a7 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:13.838065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:13.838820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:13.839272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:13.839704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] PATH_INFO: `/v3/users/be99368c8eb04a1891399dcb4bdba2c5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:13.841075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=be99368c8eb04a1891399dcb4bdba2c5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:13.843048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:13.916859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be99368c8eb04a1891399dcb4bdba2c5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.918279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be99368c8eb04a1891399dcb4bdba2c5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.932179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be99368c8eb04a1891399dcb4bdba2c5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.948364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be99368c8eb04a1891399dcb4bdba2c5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.949232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be99368c8eb04a1891399dcb4bdba2c5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:13.955703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:13.974613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1029/2064] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:28:13 2022] DELETE /identity/v3/users/be99368c8eb04a1891399dcb4bdba2c5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:14.045970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad0fe-cb78-4237-8cb8-cf98f716dc6d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:14.046584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad0fe-cb78-4237-8cb8-cf98f716dc6d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:14.047095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad0fe-cb78-4237-8cb8-cf98f716dc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:14.069671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a47ad0fe-cb78-4237-8cb8-cf98f716dc6d None None] MFA Rules not processed for user `75018aeecd6843c0a75bc95e8757fd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:14.071075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a47ad0fe-cb78-4237-8cb8-cf98f716dc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:14.147191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1036/2065] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:14.477956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:14.480186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:14.649790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9665fff-ccbb-4009-a49c-c2d68b0a64ce admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:14.660462 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:14.662724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:14.662993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:14.663390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] PATH_INFO: `/v3/projects/7af8b09212634a04aa1893e372fcc4a6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:14.665184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af8b09212634a04aa1893e372fcc4a6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:14.668874 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:14.727744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af8b09212634a04aa1893e372fcc4a6', 'request_id': 'req-d56e2857-7c15-4632-b824-06a7be6bcc7a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:14.736836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af8b09212634a04aa1893e372fcc4a6', 'request_id': 'req-d56e2857-7c15-4632-b824-06a7be6bcc7a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:14.744438 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af8b09212634a04aa1893e372fcc4a6', 'request_id': 'req-d56e2857-7c15-4632-b824-06a7be6bcc7a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:14.765694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] The token cache is being invalidate because project 7af8b09212634a04aa1893e372fcc4a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:28:14.766844 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:14.767669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d56e2857-7c15-4632-b824-06a7be6bcc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:14.770614 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1030/2066] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:28:14 2022] DELETE /identity/v3/projects/7af8b09212634a04aa1893e372fcc4a6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:14.777491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a4349-c0b6-401d-af7c-e722fd700707 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:14.777779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a4349-c0b6-401d-af7c-e722fd700707 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:14.778054 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a4349-c0b6-401d-af7c-e722fd700707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:14.810720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:14.829895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e25a4349-c0b6-401d-af7c-e722fd700707 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:14.831835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e25a4349-c0b6-401d-af7c-e722fd700707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:14.911488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1037/2067] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:28:14.921481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c481f-e0b9-4b53-8865-9b6d28532a7d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:14.922584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c481f-e0b9-4b53-8865-9b6d28532a7d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:14.923089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c481f-e0b9-4b53-8865-9b6d28532a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:14.950146 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7a1c481f-e0b9-4b53-8865-9b6d28532a7d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:14.951503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7a1c481f-e0b9-4b53-8865-9b6d28532a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:14.969611 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1031/2068] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:14.971205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a4349-c0b6-401d-af7c-e722fd700707 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:14.987157 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a4349-c0b6-401d-af7c-e722fd700707 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:14.995917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:14.996696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:15.000600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:15.000600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:15.000600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:28:15.000600 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:15.001575 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:15.025801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1038/2069] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:28:15.028582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c481f-e0b9-4b53-8865-9b6d28532a7d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:15.040556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c481f-e0b9-4b53-8865-9b6d28532a7d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:15.060362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41239326-6237-44f0-add5-e8e288780cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:15.065762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41239326-6237-44f0-add5-e8e288780cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:15.066623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41239326-6237-44f0-add5-e8e288780cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:15.067195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41239326-6237-44f0-add5-e8e288780cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:15.068131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41239326-6237-44f0-add5-e8e288780cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:15.070366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41239326-6237-44f0-add5-e8e288780cfa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:15.138379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1032/2070] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:28:15 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:28:15.141499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:15.161002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60231b4-e7c2-4228-be94-121344a0e9ea admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:15.261438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a702fc-f70d-49da-b629-dadb4e82db7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:15.262137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a702fc-f70d-49da-b629-dadb4e82db7b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:15.262679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a702fc-f70d-49da-b629-dadb4e82db7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:15.263022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a702fc-f70d-49da-b629-dadb4e82db7b admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:15.263732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a702fc-f70d-49da-b629-dadb4e82db7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:15.266367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a702fc-f70d-49da-b629-dadb4e82db7b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:15.298379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1039/2071] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:28:15 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:28:15.303283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41239326-6237-44f0-add5-e8e288780cfa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:15.321263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41239326-6237-44f0-add5-e8e288780cfa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:15.327047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086c032-ec84-4c42-8819-9278251018c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:15.327664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c032-ec84-4c42-8819-9278251018c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:15.327938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c032-ec84-4c42-8819-9278251018c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:15.328205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c032-ec84-4c42-8819-9278251018c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:15.329098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086c032-ec84-4c42-8819-9278251018c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:15.330507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086c032-ec84-4c42-8819-9278251018c6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:15.346161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1033/2072] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:28:15.347415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a702fc-f70d-49da-b629-dadb4e82db7b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:15.367751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a702fc-f70d-49da-b629-dadb4e82db7b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:15.372734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff76eb-f0af-407e-86c8-c0d7137c1b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:15.373351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff76eb-f0af-407e-86c8-c0d7137c1b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:15.374277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff76eb-f0af-407e-86c8-c0d7137c1b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:15.374277 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff76eb-f0af-407e-86c8-c0d7137c1b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:15.374747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff76eb-f0af-407e-86c8-c0d7137c1b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:15.376563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff76eb-f0af-407e-86c8-c0d7137c1b82 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:15.390376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1040/2073] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:28:15.395052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086c032-ec84-4c42-8819-9278251018c6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:15.406416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086c032-ec84-4c42-8819-9278251018c6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:15.411102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:15.411764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:15.412036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:15.412306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] PATH_INFO: `/v3/projects/0131e3041de74f4eac7e74f61d76c671/users/af1b8d880aa0443395094d11b64a9100/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:15.430213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0131e3041de74f4eac7e74f61d76c671, user_id=af1b8d880aa0443395094d11b64a9100, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:15.433182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:15.456897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1034/2074] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:28:15 2022] PUT /identity/v3/projects/0131e3041de74f4eac7e74f61d76c671/users/af1b8d880aa0443395094d11b64a9100/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:15.462243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e2e82-c9a2-4e25-b80c-2cc478f00738 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:15.463254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e2e82-c9a2-4e25-b80c-2cc478f00738 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:15.463625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e2e82-c9a2-4e25-b80c-2cc478f00738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:15.496606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c27e2e82-c9a2-4e25-b80c-2cc478f00738 None None] MFA Rules not processed for user `af1b8d880aa0443395094d11b64a9100`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:15.497883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c27e2e82-c9a2-4e25-b80c-2cc478f00738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:15.562864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:15.567288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1041/2075] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:15.581700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:15.583517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:15.610386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:15.643003 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8f0a-d673-45f7-8b72-e4d42dba45a7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:15.647398 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e083243-b652-402a-afe5-c2bfcf73499c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:15.648058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e083243-b652-402a-afe5-c2bfcf73499c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:15.648328 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e083243-b652-402a-afe5-c2bfcf73499c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:15.648594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e083243-b652-402a-afe5-c2bfcf73499c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:15.658069 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e083243-b652-402a-afe5-c2bfcf73499c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:15.659508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e083243-b652-402a-afe5-c2bfcf73499c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:15.668630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1035/2076] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:18.793855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e2e82-c9a2-4e25-b80c-2cc478f00738 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:18.860175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e2e82-c9a2-4e25-b80c-2cc478f00738 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:18.865605 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:18.866443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:18.866722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:18.866993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] PATH_INFO: `/v3/users/af1b8d880aa0443395094d11b64a9100` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:18.868152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1b8d880aa0443395094d11b64a9100)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:18.870416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:18.923657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1b8d880aa0443395094d11b64a9100'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:18.924542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1b8d880aa0443395094d11b64a9100'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:18.936755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1b8d880aa0443395094d11b64a9100'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:18.949021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1b8d880aa0443395094d11b64a9100'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:18.949646 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1b8d880aa0443395094d11b64a9100'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:18.963866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1042/2077] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:28:18 2022] DELETE /identity/v3/users/af1b8d880aa0443395094d11b64a9100 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:18.968314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56c47-8f7f-4de3-9df6-1c725d332f5a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:18.968843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56c47-8f7f-4de3-9df6-1c725d332f5a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:18.968843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56c47-8f7f-4de3-9df6-1c725d332f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:18.990167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0ca56c47-8f7f-4de3-9df6-1c725d332f5a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:18.991462 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0ca56c47-8f7f-4de3-9df6-1c725d332f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:19.076941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1036/2078] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:19.089163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:19.090023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:19.156889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a272-df00-4d07-8d62-d96386c86de7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:19.160956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a92c66-b677-49ca-bd92-0822c45675da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:19.161584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a92c66-b677-49ca-bd92-0822c45675da service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.161868 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a92c66-b677-49ca-bd92-0822c45675da service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.162140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a92c66-b677-49ca-bd92-0822c45675da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.173025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a92c66-b677-49ca-bd92-0822c45675da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:19.174518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a92c66-b677-49ca-bd92-0822c45675da service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:19.186405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1043/2079] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:19.467973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca56c47-8f7f-4de3-9df6-1c725d332f5a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:19.469651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0ca56c47-8f7f-4de3-9df6-1c725d332f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:19.482892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca56c47-8f7f-4de3-9df6-1c725d332f5a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:19.487571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:19.488194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.488471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.488742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] PATH_INFO: `/v3/projects/0131e3041de74f4eac7e74f61d76c671` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.489916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0131e3041de74f4eac7e74f61d76c671)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:19.491799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:19.541586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0131e3041de74f4eac7e74f61d76c671', 'request_id': 'req-f2619447-3d87-40a6-a87d-e5f64661b89b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:19.548947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0131e3041de74f4eac7e74f61d76c671', 'request_id': 'req-f2619447-3d87-40a6-a87d-e5f64661b89b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:19.555469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0131e3041de74f4eac7e74f61d76c671', 'request_id': 'req-f2619447-3d87-40a6-a87d-e5f64661b89b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:19.567352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] The token cache is being invalidate because project 0131e3041de74f4eac7e74f61d76c671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:28:19.567635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:19.568253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2619447-3d87-40a6-a87d-e5f64661b89b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:28:19.569730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1037/2080] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:28:19 2022] DELETE /identity/v3/projects/0131e3041de74f4eac7e74f61d76c671 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:19.574702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43e019-9cee-49bd-83fc-15b9916ba9c5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.574985 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43e019-9cee-49bd-83fc-15b9916ba9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.575259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43e019-9cee-49bd-83fc-15b9916ba9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.596206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4a43e019-9cee-49bd-83fc-15b9916ba9c5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:19.597389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a43e019-9cee-49bd-83fc-15b9916ba9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:19.616568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:19.662962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1044/2081] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:19.672595 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aecb5-e062-475c-89c3-a5c89538dc06 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.672878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aecb5-e062-475c-89c3-a5c89538dc06 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.673150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aecb5-e062-475c-89c3-a5c89538dc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.693141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f63aecb5-e062-475c-89c3-a5c89538dc06 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:19.694412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f63aecb5-e062-475c-89c3-a5c89538dc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:19.702370 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1038/2082] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:19.706566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43e019-9cee-49bd-83fc-15b9916ba9c5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:19.716482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43e019-9cee-49bd-83fc-15b9916ba9c5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:19.720978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:19.721567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.721842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.722110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.722812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:28:19.723113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:19.724701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:19.737712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1045/2083] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:28:19.742011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aecb5-e062-475c-89c3-a5c89538dc06 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:19.752075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aecb5-e062-475c-89c3-a5c89538dc06 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:19.756656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0e9a-b3f0-4f40-a651-6f256671d83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:19.757237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0e9a-b3f0-4f40-a651-6f256671d83d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.757509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0e9a-b3f0-4f40-a651-6f256671d83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.757777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0e9a-b3f0-4f40-a651-6f256671d83d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.758437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb0e9a-b3f0-4f40-a651-6f256671d83d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:19.760349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb0e9a-b3f0-4f40-a651-6f256671d83d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:19.787111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1039/2084] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:28:19 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:28:19.791981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:19.802556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20a27e-c45c-43a2-8856-b84eb87e34c5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:19.862740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99036529-27ad-4391-98b5-3abaffcc314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:19.863349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99036529-27ad-4391-98b5-3abaffcc314f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.863625 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99036529-27ad-4391-98b5-3abaffcc314f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.863899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99036529-27ad-4391-98b5-3abaffcc314f admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.864569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99036529-27ad-4391-98b5-3abaffcc314f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:19.866518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99036529-27ad-4391-98b5-3abaffcc314f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:19.888922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1046/2085] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:28:19 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:28:19.893244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0e9a-b3f0-4f40-a651-6f256671d83d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:19.903897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0e9a-b3f0-4f40-a651-6f256671d83d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:19.908531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a80fcb-742e-4f99-894f-617165309bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:19.909128 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a80fcb-742e-4f99-894f-617165309bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.909407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a80fcb-742e-4f99-894f-617165309bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.909678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a80fcb-742e-4f99-894f-617165309bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.910347 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a80fcb-742e-4f99-894f-617165309bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:19.911985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a80fcb-742e-4f99-894f-617165309bd9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:19.923175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1040/2086] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:28:19.927271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99036529-27ad-4391-98b5-3abaffcc314f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:19.937756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99036529-27ad-4391-98b5-3abaffcc314f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:19.942571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cd0a5-677b-46d6-98dc-a5f0ae1eae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:19.943230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd0a5-677b-46d6-98dc-a5f0ae1eae7c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.943537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd0a5-677b-46d6-98dc-a5f0ae1eae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.943805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd0a5-677b-46d6-98dc-a5f0ae1eae7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.944442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cd0a5-677b-46d6-98dc-a5f0ae1eae7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:19.946062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cd0a5-677b-46d6-98dc-a5f0ae1eae7c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:19.957430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1047/2087] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:28:19.961625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a80fcb-742e-4f99-894f-617165309bd9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:19.972923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a80fcb-742e-4f99-894f-617165309bd9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:19.979352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:19.980010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:19.980283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:19.980551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] PATH_INFO: `/v3/projects/3696c60531db4c2ab278361ea87cdf9b/users/02f627ff8a1c4e32a640b3bea9258854/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:19.984126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:19.998626 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3696c60531db4c2ab278361ea87cdf9b, user_id=02f627ff8a1c4e32a640b3bea9258854, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:20.001299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:20.013939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1041/2088] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:28:19 2022] PUT /identity/v3/projects/3696c60531db4c2ab278361ea87cdf9b/users/02f627ff8a1c4e32a640b3bea9258854/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:28:20.025649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f952e-c451-4625-8572-95010455f241 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:20.025649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f952e-c451-4625-8572-95010455f241 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:20.025958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f952e-c451-4625-8572-95010455f241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:20.055246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e01f952e-c451-4625-8572-95010455f241 None None] MFA Rules not processed for user `02f627ff8a1c4e32a640b3bea9258854`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:20.056986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e01f952e-c451-4625-8572-95010455f241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:20.125697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1048/2089] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:20.137765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:20.139540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:20.198590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67894f-dd29-4dea-869d-9db5642f13a5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:20.202201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783f7c-1741-437d-918b-83230069230d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ac934ed0122470cb489d6b324c664f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:20.202829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783f7c-1741-437d-918b-83230069230d service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:20.203098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783f7c-1741-437d-918b-83230069230d service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:20.203362 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783f7c-1741-437d-918b-83230069230d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:20.211832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4783f7c-1741-437d-918b-83230069230d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:20.214914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4783f7c-1741-437d-918b-83230069230d service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:20.223698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1042/2090] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:30.122724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:30.219551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:39.034151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee6c4-309d-487a-98ee-4be38c4bd726 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:39.034628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee6c4-309d-487a-98ee-4be38c4bd726 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:39.034758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee6c4-309d-487a-98ee-4be38c4bd726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:39.064033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-dceee6c4-309d-487a-98ee-4be38c4bd726 None None] MFA Rules not processed for user `6b4b2700c20946ea93a1d20d7dea929f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:28:39.065278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dceee6c4-309d-487a-98ee-4be38c4bd726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:39.127829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1049/2091] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:28:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:28:40.258561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783f7c-1741-437d-918b-83230069230d service glance] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:28:40.260071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d4783f7c-1741-437d-918b-83230069230d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:40.331404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783f7c-1741-437d-918b-83230069230d service glance] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:28:40.336214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea87cb-f3a6-424b-9585-e2cade5a6655 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:28:40.336802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea87cb-f3a6-424b-9585-e2cade5a6655 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:28:40.337075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea87cb-f3a6-424b-9585-e2cade5a6655 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:28:40.337339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea87cb-f3a6-424b-9585-e2cade5a6655 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:28:40.339226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-37ea87cb-f3a6-424b-9585-e2cade5a6655 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:28:40.351062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea87cb-f3a6-424b-9585-e2cade5a6655 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:28:40.352608 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea87cb-f3a6-424b-9585-e2cade5a6655 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:28:40.363307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1043/2092] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:28:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:28:49.120131 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=25 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:28:50.359080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=32 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:29:09.209952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceee6c4-309d-487a-98ee-4be38c4bd726 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:09.272280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceee6c4-309d-487a-98ee-4be38c4bd726 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:09.276674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:09.277293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:09.277567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:09.277836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] PATH_INFO: `/v3/users/02f627ff8a1c4e32a640b3bea9258854` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:09.279006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02f627ff8a1c4e32a640b3bea9258854)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:09.280930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:09.326468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02f627ff8a1c4e32a640b3bea9258854'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.327114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02f627ff8a1c4e32a640b3bea9258854'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.339577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02f627ff8a1c4e32a640b3bea9258854'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.347814 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02f627ff8a1c4e32a640b3bea9258854'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.348750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02f627ff8a1c4e32a640b3bea9258854'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.363135 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1050/2093] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:29:09 2022] DELETE /identity/v3/users/02f627ff8a1c4e32a640b3bea9258854 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:29:09.367607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17657-65c5-465b-a239-7de3ca78ed03 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:09.368063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17657-65c5-465b-a239-7de3ca78ed03 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:09.368200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17657-65c5-465b-a239-7de3ca78ed03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:09.386845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-55c17657-65c5-465b-a239-7de3ca78ed03 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:09.388027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-55c17657-65c5-465b-a239-7de3ca78ed03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:09.446606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1044/2094] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:09.456167 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:09.457547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:09.518673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2239dc-9cf7-4d9a-a97f-33d100c22e67 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:09.522198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc1313-7cff-4283-b146-162a21d9ef45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:09.522821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1313-7cff-4283-b146-162a21d9ef45 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:09.523099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1313-7cff-4283-b146-162a21d9ef45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:09.523364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1313-7cff-4283-b146-162a21d9ef45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:09.532481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc1313-7cff-4283-b146-162a21d9ef45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:09.533902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc1313-7cff-4283-b146-162a21d9ef45 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:09.542978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1051/2095] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:29:09.820953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17657-65c5-465b-a239-7de3ca78ed03 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:09.822554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-55c17657-65c5-465b-a239-7de3ca78ed03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:09.835518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17657-65c5-465b-a239-7de3ca78ed03 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:09.840254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:09.840905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:09.841177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:09.841548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] PATH_INFO: `/v3/projects/3696c60531db4c2ab278361ea87cdf9b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:09.843638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3696c60531db4c2ab278361ea87cdf9b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:09.845536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:09.892390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3696c60531db4c2ab278361ea87cdf9b', 'request_id': 'req-05959ded-18d0-4066-940b-1c4658af19a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.898550 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3696c60531db4c2ab278361ea87cdf9b', 'request_id': 'req-05959ded-18d0-4066-940b-1c4658af19a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.905177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3696c60531db4c2ab278361ea87cdf9b', 'request_id': 'req-05959ded-18d0-4066-940b-1c4658af19a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.916851 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] The token cache is being invalidate because project 3696c60531db4c2ab278361ea87cdf9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:29:09.917140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.917754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-05959ded-18d0-4066-940b-1c4658af19a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:09.919272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1045/2096] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:29:09 2022] DELETE /identity/v3/projects/3696c60531db4c2ab278361ea87cdf9b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:29:09.924428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a676a2-7989-42e5-86b0-f838906f1d38 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:09.924713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a676a2-7989-42e5-86b0-f838906f1d38 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:09.925012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a676a2-7989-42e5-86b0-f838906f1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:09.944308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c1a676a2-7989-42e5-86b0-f838906f1d38 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:09.945489 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c1a676a2-7989-42e5-86b0-f838906f1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:09.962540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:29:10.006050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1052/2097] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:10.015557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656464d4-78d1-4b17-bca3-71c0d70ed2e9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.015862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656464d4-78d1-4b17-bca3-71c0d70ed2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.016116 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656464d4-78d1-4b17-bca3-71c0d70ed2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.036796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-656464d4-78d1-4b17-bca3-71c0d70ed2e9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:10.037977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-656464d4-78d1-4b17-bca3-71c0d70ed2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:10.046961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1046/2098] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:10.051125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a676a2-7989-42e5-86b0-f838906f1d38 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:10.060652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a676a2-7989-42e5-86b0-f838906f1d38 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:10.065208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:10.065804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.066079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.066367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.067011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:29:10.067305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:10.068898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:10.086198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1053/2099] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:29:10.087318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656464d4-78d1-4b17-bca3-71c0d70ed2e9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:10.096911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656464d4-78d1-4b17-bca3-71c0d70ed2e9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:10.101677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eeb6b-49b0-4920-8f7a-1f7f3eda28f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:10.102269 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eeb6b-49b0-4920-8f7a-1f7f3eda28f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.102576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eeb6b-49b0-4920-8f7a-1f7f3eda28f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.102839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eeb6b-49b0-4920-8f7a-1f7f3eda28f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.103464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648eeb6b-49b0-4920-8f7a-1f7f3eda28f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:10.105582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648eeb6b-49b0-4920-8f7a-1f7f3eda28f0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:10.134879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1047/2100] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:29:10 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:29:10.139953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:10.150151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009265f-7c10-4e74-82e1-d9591eb54276 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:10.206809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d372c-00da-4409-b7a5-5014769831e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:10.207404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d372c-00da-4409-b7a5-5014769831e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.207677 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d372c-00da-4409-b7a5-5014769831e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.207942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d372c-00da-4409-b7a5-5014769831e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.208569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d372c-00da-4409-b7a5-5014769831e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:10.210530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d372c-00da-4409-b7a5-5014769831e1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:10.230693 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1054/2101] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:29:10 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:29:10.234776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eeb6b-49b0-4920-8f7a-1f7f3eda28f0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:10.244917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eeb6b-49b0-4920-8f7a-1f7f3eda28f0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:10.250607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac4d3f-15d7-48ea-999e-ffce5d91ed57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:10.251130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac4d3f-15d7-48ea-999e-ffce5d91ed57 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.251410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac4d3f-15d7-48ea-999e-ffce5d91ed57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.251674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac4d3f-15d7-48ea-999e-ffce5d91ed57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.252302 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac4d3f-15d7-48ea-999e-ffce5d91ed57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:10.253870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac4d3f-15d7-48ea-999e-ffce5d91ed57 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:10.264495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1048/2102] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:29:10.268205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d372c-00da-4409-b7a5-5014769831e1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:10.279552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d372c-00da-4409-b7a5-5014769831e1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:10.285446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740d68c-76ac-4cc3-b043-5f7ff61f1223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:10.286123 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740d68c-76ac-4cc3-b043-5f7ff61f1223 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.286444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740d68c-76ac-4cc3-b043-5f7ff61f1223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.286719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740d68c-76ac-4cc3-b043-5f7ff61f1223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.287552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740d68c-76ac-4cc3-b043-5f7ff61f1223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:10.289359 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740d68c-76ac-4cc3-b043-5f7ff61f1223 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:10.300392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1055/2103] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:29:10.304781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac4d3f-15d7-48ea-999e-ffce5d91ed57 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:10.315555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac4d3f-15d7-48ea-999e-ffce5d91ed57 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:10.320127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:10.320762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.321034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.321299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] PATH_INFO: `/v3/projects/42e2bfadfd7546eda906a580a16c5a59/users/800ab212b7cb45a59949f2b142faeeb7/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.336633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e2bfadfd7546eda906a580a16c5a59, user_id=800ab212b7cb45a59949f2b142faeeb7, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:10.339313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:10.352040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1049/2104] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:29:10 2022] PUT /identity/v3/projects/42e2bfadfd7546eda906a580a16c5a59/users/800ab212b7cb45a59949f2b142faeeb7/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:29:10.363153 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bed2c-d0a5-4c3e-8272-0ccb29f05c51 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.363412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bed2c-d0a5-4c3e-8272-0ccb29f05c51 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.363701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bed2c-d0a5-4c3e-8272-0ccb29f05c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.391847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6e9bed2c-d0a5-4c3e-8272-0ccb29f05c51 None None] MFA Rules not processed for user `800ab212b7cb45a59949f2b142faeeb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:10.393027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6e9bed2c-d0a5-4c3e-8272-0ccb29f05c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:10.458068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1056/2105] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:29:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:10.468549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:10.470218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:10.492230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:29:10.530346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa672d-862c-41fa-8dab-3e4501250bd0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:10.533930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:10.534557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:10.534835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:10.535123 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:10.543448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:10.545597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:10.554512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1050/2106] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:29:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:29:11.934799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9736b-1f39-42dc-884b-f89731268547 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:11.935369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9736b-1f39-42dc-884b-f89731268547 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:11.935369 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9736b-1f39-42dc-884b-f89731268547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:11.953952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-61b9736b-1f39-42dc-884b-f89731268547 None None] MFA Rules not processed for user `800ab212b7cb45a59949f2b142faeeb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:11.955189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-61b9736b-1f39-42dc-884b-f89731268547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:11.961996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1057/2107] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:20.550622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=17 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:29:21.948859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:29:35.638233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:35.639779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:35.696390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212094d-d70c-41bd-90dc-2177f8ce52a5 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:35.699935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:35.700526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:35.700798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:35.701067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:35.702798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:35.776785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:35.778216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:35.789827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1051/2108] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:29:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:29:36.125636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9736b-1f39-42dc-884b-f89731268547 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:36.127278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-61b9736b-1f39-42dc-884b-f89731268547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:36.140125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9736b-1f39-42dc-884b-f89731268547 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:36.144539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97dda8-be04-46eb-9774-53ccc0edf247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:36.145142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97dda8-be04-46eb-9774-53ccc0edf247 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:36.145407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97dda8-be04-46eb-9774-53ccc0edf247 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:36.145676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97dda8-be04-46eb-9774-53ccc0edf247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:36.147542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ee97dda8-be04-46eb-9774-53ccc0edf247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:36.234272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97dda8-be04-46eb-9774-53ccc0edf247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:36.235857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97dda8-be04-46eb-9774-53ccc0edf247 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:36.246599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1058/2109] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:29:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:29:38.557353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:38.567496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e768e4-9841-433a-abf6-571095aa5904 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:38.571208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:38.571802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:38.572073 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:38.572340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:38.574076 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:38.588761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:38.590184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:38.601639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1052/2110] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:29:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:29:46.201533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174bc9f-9d85-4e11-9857-c67a1b2251b2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:46.202027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174bc9f-9d85-4e11-9857-c67a1b2251b2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:46.202125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174bc9f-9d85-4e11-9857-c67a1b2251b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:46.221725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9174bc9f-9d85-4e11-9857-c67a1b2251b2 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:46.222942 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9174bc9f-9d85-4e11-9857-c67a1b2251b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:46.278430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1059/2111] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:46.319044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:46.320524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:46.389352 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe14af-0890-4d9d-b4dd-134afd2174b8 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:46.394983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07174d-923e-4baf-aa2e-1ec98c5ebebc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:46.395511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07174d-923e-4baf-aa2e-1ec98c5ebebc service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:46.395781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07174d-923e-4baf-aa2e-1ec98c5ebebc service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:46.396046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07174d-923e-4baf-aa2e-1ec98c5ebebc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:46.403778 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07174d-923e-4baf-aa2e-1ec98c5ebebc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:46.405338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07174d-923e-4baf-aa2e-1ec98c5ebebc service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:46.413646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1053/2112] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:29:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:29:46.939872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2078-19b1-4865-98bf-93976e3a08db None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:46.940159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2078-19b1-4865-98bf-93976e3a08db None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:46.940428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2078-19b1-4865-98bf-93976e3a08db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:46.959847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-512c2078-19b1-4865-98bf-93976e3a08db None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:46.961051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-512c2078-19b1-4865-98bf-93976e3a08db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:46.968470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1060/2113] 10.222.0.16 () {58 vars in 967 bytes} [Mon Oct 17 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:29:46.982109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07174d-923e-4baf-aa2e-1ec98c5ebebc service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:46.993203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07174d-923e-4baf-aa2e-1ec98c5ebebc service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:47.009281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f905616-12e9-4fc4-bcf7-6a06c406910d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:47.009897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f905616-12e9-4fc4-bcf7-6a06c406910d service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:47.010171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f905616-12e9-4fc4-bcf7-6a06c406910d service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:47.010474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f905616-12e9-4fc4-bcf7-6a06c406910d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:47.019431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f905616-12e9-4fc4-bcf7-6a06c406910d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:47.020990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f905616-12e9-4fc4-bcf7-6a06c406910d service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:47.029119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1054/2114] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:29:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:29:47.273405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9908-1e78-4d9d-aed6-8222d645b2ae None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:47.273903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9908-1e78-4d9d-aed6-8222d645b2ae None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:47.274001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9908-1e78-4d9d-aed6-8222d645b2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:47.292703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-157a9908-1e78-4d9d-aed6-8222d645b2ae None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:47.293882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-157a9908-1e78-4d9d-aed6-8222d645b2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:47.301604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1061/2115] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:29:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:47.314378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f905616-12e9-4fc4-bcf7-6a06c406910d service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:47.320635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:29:47.325416 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f905616-12e9-4fc4-bcf7-6a06c406910d service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:47.330947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee2b3-9ce8-4256-a2d9-91cab287aa96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:47.331534 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee2b3-9ce8-4256-a2d9-91cab287aa96 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:47.331806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee2b3-9ce8-4256-a2d9-91cab287aa96 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:47.332074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee2b3-9ce8-4256-a2d9-91cab287aa96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:47.340716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee2b3-9ce8-4256-a2d9-91cab287aa96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:47.342287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee2b3-9ce8-4256-a2d9-91cab287aa96 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:47.350583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1055/2116] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:29:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:29:47.673770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765692c5-212c-4029-8f3f-154c30754112 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:47.674048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765692c5-212c-4029-8f3f-154c30754112 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:47.674351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765692c5-212c-4029-8f3f-154c30754112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:47.693878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-765692c5-212c-4029-8f3f-154c30754112 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:47.695013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-765692c5-212c-4029-8f3f-154c30754112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:47.702098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1062/2117] 10.222.0.16 () {58 vars in 967 bytes} [Mon Oct 17 10:29:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:47.714843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee2b3-9ce8-4256-a2d9-91cab287aa96 service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:47.726334 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee2b3-9ce8-4256-a2d9-91cab287aa96 service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:47.731855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff0ec3-7f98-40b6-9bcf-b2dc6f574036 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:47.732437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff0ec3-7f98-40b6-9bcf-b2dc6f574036 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:47.732714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff0ec3-7f98-40b6-9bcf-b2dc6f574036 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:47.732986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff0ec3-7f98-40b6-9bcf-b2dc6f574036 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:47.741226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff0ec3-7f98-40b6-9bcf-b2dc6f574036 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:47.742842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff0ec3-7f98-40b6-9bcf-b2dc6f574036 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:47.751639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1056/2118] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:29:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 10:29:49.865263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765692c5-212c-4029-8f3f-154c30754112 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:49.927080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765692c5-212c-4029-8f3f-154c30754112 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:49.931449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:49.932063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:49.932337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:49.932603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] PATH_INFO: `/v3/users/800ab212b7cb45a59949f2b142faeeb7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:49.933694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=800ab212b7cb45a59949f2b142faeeb7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:49.935639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:49.957862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:29:49.981976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '800ab212b7cb45a59949f2b142faeeb7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:49.982846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '800ab212b7cb45a59949f2b142faeeb7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:49.994899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '800ab212b7cb45a59949f2b142faeeb7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:50.003271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '800ab212b7cb45a59949f2b142faeeb7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:50.003895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '800ab212b7cb45a59949f2b142faeeb7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:50.016617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1063/2119] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:29:49 2022] DELETE /identity/v3/users/800ab212b7cb45a59949f2b142faeeb7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:29:50.021037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cdf0c-a02c-4b7a-8d8b-ee5584f979cc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:50.021512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cdf0c-a02c-4b7a-8d8b-ee5584f979cc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:50.021627 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cdf0c-a02c-4b7a-8d8b-ee5584f979cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:50.040936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-351cdf0c-a02c-4b7a-8d8b-ee5584f979cc None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:29:50.042122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-351cdf0c-a02c-4b7a-8d8b-ee5584f979cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:50.101491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1057/2120] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:29:50.111798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:50.113198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:50.170965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ac7ef-18be-407c-9d4d-bb6c9d96d923 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:50.175671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725e39f-a6ad-45ce-b757-109ca3bf677e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:50.176259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e39f-a6ad-45ce-b757-109ca3bf677e service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:50.176532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e39f-a6ad-45ce-b757-109ca3bf677e service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:50.176798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e39f-a6ad-45ce-b757-109ca3bf677e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:50.185475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725e39f-a6ad-45ce-b757-109ca3bf677e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:50.186923 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725e39f-a6ad-45ce-b757-109ca3bf677e service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:50.197151 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1064/2121] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:29:50.370801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cdf0c-a02c-4b7a-8d8b-ee5584f979cc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:50.372473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-351cdf0c-a02c-4b7a-8d8b-ee5584f979cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:29:50.391053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cdf0c-a02c-4b7a-8d8b-ee5584f979cc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:29:50.396827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:29:50.397584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:29:50.398071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:29:50.398431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] PATH_INFO: `/v3/projects/42e2bfadfd7546eda906a580a16c5a59` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:29:50.399497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e2bfadfd7546eda906a580a16c5a59)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:29:50.401389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:29:50.449522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e2bfadfd7546eda906a580a16c5a59', 'request_id': 'req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:50.456238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e2bfadfd7546eda906a580a16c5a59', 'request_id': 'req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:50.461843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e2bfadfd7546eda906a580a16c5a59', 'request_id': 'req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:50.473429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] The token cache is being invalidate because project 42e2bfadfd7546eda906a580a16c5a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:29:50.473718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:50.474344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:29:50.475831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1058/2122] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:29:50 2022] DELETE /identity/v3/projects/42e2bfadfd7546eda906a580a16c5a59 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:29:59.983337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725e39f-a6ad-45ce-b757-109ca3bf677e service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:29:59.992550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725e39f-a6ad-45ce-b757-109ca3bf677e service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:30:00.048408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:30:00.049011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:00.049291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:00.049561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:00.051366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:00.060765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:30:00.062253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:30:00.071130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1065/2123] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:29:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:30:00.470537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:30:05.344448 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:30:05.345882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:05.418818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb813d-9f7e-4cef-8b2e-e6c671cdb79d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:30:05.424475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:30:05.425081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:05.425382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:05.425651 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:05.427809 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:05.485084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:30:05.486692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:30:05.495515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1059/2124] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:30:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:30:10.067399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=7,keystone:SELECT=72,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:30:15.491361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=56 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:30:27.749584 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:30:27.751262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:27.769265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75947a97-5c69-446c-b4a7-6ded7a2ce5dc service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:30:27.775386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:30:27.776075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:27.776189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:27.776454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:27.778177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:27.838539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:30:27.840132 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:30:27.849050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1066/2125] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:30:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:30:31.125345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:30:31.151992 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4abff0-e189-44b7-95af-144e033de046 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:30:31.168118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aa2ad-7d1a-438e-af1b-b86ae60e06bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:30:31.168805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa2ad-7d1a-438e-af1b-b86ae60e06bd service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:31.169264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa2ad-7d1a-438e-af1b-b86ae60e06bd service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:31.182514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa2ad-7d1a-438e-af1b-b86ae60e06bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:31.188615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f67aa2ad-7d1a-438e-af1b-b86ae60e06bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:31.231441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aa2ad-7d1a-438e-af1b-b86ae60e06bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:30:31.235572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aa2ad-7d1a-438e-af1b-b86ae60e06bd service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:30:31.251419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1060/2126] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:30:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:30:37.440754 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:30:37.453499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9ad68-f6f6-439b-b8b1-ee499b69884f service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:30:37.459140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbf49d-9762-465f-909a-bc712f1f361e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:30:37.459881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbf49d-9762-465f-909a-bc712f1f361e service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:37.460159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbf49d-9762-465f-909a-bc712f1f361e service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:37.460430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbf49d-9762-465f-909a-bc712f1f361e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:37.462449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-56fbf49d-9762-465f-909a-bc712f1f361e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:37.522730 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbf49d-9762-465f-909a-bc712f1f361e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:30:37.524302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbf49d-9762-465f-909a-bc712f1f361e service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:30:37.535909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1067/2127] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:30:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:30:41.244263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:30:42.207539 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b2ea8-573b-4a78-8cb2-4ff0b1ee0449 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:42.207823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b2ea8-573b-4a78-8cb2-4ff0b1ee0449 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:42.208097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b2ea8-573b-4a78-8cb2-4ff0b1ee0449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:42.228480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f21b2ea8-573b-4a78-8cb2-4ff0b1ee0449 None None] MFA Rules not processed for user `aaedb0e1920a42eb97e8dc2f3ec76905`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:30:42.229933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f21b2ea8-573b-4a78-8cb2-4ff0b1ee0449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:42.238530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1061/2128] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:30:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:30:45.440491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3cfcd-f8a4-413f-9418-280608209cfa None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:45.440941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3cfcd-f8a4-413f-9418-280608209cfa None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:45.441051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3cfcd-f8a4-413f-9418-280608209cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:45.460018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-44d3cfcd-f8a4-413f-9418-280608209cfa None None] MFA Rules not processed for user `6b4b2700c20946ea93a1d20d7dea929f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:30:45.461256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-44d3cfcd-f8a4-413f-9418-280608209cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:45.468888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1068/2129] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:30:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:30:45.773371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b2ea8-573b-4a78-8cb2-4ff0b1ee0449 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:30:45.774848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f21b2ea8-573b-4a78-8cb2-4ff0b1ee0449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:45.786526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b2ea8-573b-4a78-8cb2-4ff0b1ee0449 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:30:45.790818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:30:45.791275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:45.791546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:45.791811 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:45.793453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:45.802423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:30:45.803827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:30:45.811920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1062/2130] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:30:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:30:46.397393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3cfcd-f8a4-413f-9418-280608209cfa None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:30:46.398984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-44d3cfcd-f8a4-413f-9418-280608209cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:46.499585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3cfcd-f8a4-413f-9418-280608209cfa None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:30:46.504497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96f89c-fe7b-4180-a2f2-783f10c378cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:30:46.505099 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96f89c-fe7b-4180-a2f2-783f10c378cf service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:46.505370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96f89c-fe7b-4180-a2f2-783f10c378cf service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:46.505638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96f89c-fe7b-4180-a2f2-783f10c378cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:46.507409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5e96f89c-fe7b-4180-a2f2-783f10c378cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:46.520159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96f89c-fe7b-4180-a2f2-783f10c378cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:30:46.521686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96f89c-fe7b-4180-a2f2-783f10c378cf service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:30:46.532093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1069/2131] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:30:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:30:55.808402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=9 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:30:56.527764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:30:58.611310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:30:58.626022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7f4a4-e5b3-4591-9013-9c9d1c3fe1ca service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:30:58.632036 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:30:58.632677 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:58.632957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:58.633238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:58.636040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:58.651680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:30:58.653463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:30:58.670947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1063/2132] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:30:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:30:59.393201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949af43-ae10-4b7f-ac0a-e2996445508c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:30:59.393679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949af43-ae10-4b7f-ac0a-e2996445508c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:30:59.393773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949af43-ae10-4b7f-ac0a-e2996445508c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:30:59.412882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e949af43-ae10-4b7f-ac0a-e2996445508c None None] MFA Rules not processed for user `a5b7190649694b9882ab34c0f3114a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:30:59.414065 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e949af43-ae10-4b7f-ac0a-e2996445508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:30:59.467909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1070/2133] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:30:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:31:08.662515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:31:09.461038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=25 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:31:10.901496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:10.910680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318110-1099-43b0-ad80-5f2761f6e967 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:10.914169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132ff5b-8006-4bbb-9cdf-0bfc49a5277c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:10.914784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132ff5b-8006-4bbb-9cdf-0bfc49a5277c service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:10.915058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132ff5b-8006-4bbb-9cdf-0bfc49a5277c service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:10.915332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132ff5b-8006-4bbb-9cdf-0bfc49a5277c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:10.917231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d132ff5b-8006-4bbb-9cdf-0bfc49a5277c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:10.928299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132ff5b-8006-4bbb-9cdf-0bfc49a5277c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:10.929835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132ff5b-8006-4bbb-9cdf-0bfc49a5277c service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:10.939583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1064/2134] 10.222.0.16 () {60 vars in 1396 bytes} [Mon Oct 17 10:31:10 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:31:20.008113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949af43-ae10-4b7f-ac0a-e2996445508c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:20.021934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949af43-ae10-4b7f-ac0a-e2996445508c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:20.028863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:20.029474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:20.029749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:20.030490 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:20.031797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:20.046291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:20.047927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:20.068753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1071/2135] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:31:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:31:20.935350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:31:29.828080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e081f-645e-4c88-ab03-2ce94d6b4aa9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:29.828697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e081f-645e-4c88-ab03-2ce94d6b4aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:29.828697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e081f-645e-4c88-ab03-2ce94d6b4aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:29.848867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5a9e081f-645e-4c88-ab03-2ce94d6b4aa9 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:31:29.850131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5a9e081f-645e-4c88-ab03-2ce94d6b4aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:29.905177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1065/2136] 10.222.0.16 () {58 vars in 967 bytes} [Mon Oct 17 10:31:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:31:29.950580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:29.952001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:30.021975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b9700-8fd6-4a0f-a1cf-ee9003783314 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:30.028077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c392f58-b907-47fb-acf2-d5adbd43390f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:30.028664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c392f58-b907-47fb-acf2-d5adbd43390f service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:30.028941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c392f58-b907-47fb-acf2-d5adbd43390f service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:30.029206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c392f58-b907-47fb-acf2-d5adbd43390f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:30.041201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c392f58-b907-47fb-acf2-d5adbd43390f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:30.042808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c392f58-b907-47fb-acf2-d5adbd43390f service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:30.050990 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1072/2137] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:31:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:31:33.693737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e081f-645e-4c88-ab03-2ce94d6b4aa9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:33.707110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e081f-645e-4c88-ab03-2ce94d6b4aa9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:33.714230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:33.715067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:33.715816 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:33.716364 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:33.718261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:33.740932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:33.750789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:33.767950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1066/2138] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:31:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:31:35.336397 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c392f58-b907-47fb-acf2-d5adbd43390f service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:35.347210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c392f58-b907-47fb-acf2-d5adbd43390f service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:35.351423 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:35.352110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:35.352984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:35.353406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:35.355610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:35.422276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:35.424027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:35.436588 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1073/2139] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:31:36.975949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:36.987153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca30e79-9b72-4a6d-8acb-215a764a3282 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:36.992202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:36.992859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:36.993857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:36.994244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:36.995981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:37.008678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:37.010309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:37.020588 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1067/2140] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:31:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:31:37.047040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:37.048588 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:37.062762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71814c-3282-4c30-baea-dfcd7db243db service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:37.067587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:37.068289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:37.068569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:37.068839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] PATH_INFO: `/v3/users/973791f2609b4b55aa986e9fd9baceec` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:37.070106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=973791f2609b4b55aa986e9fd9baceec)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:37.072204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:37.126146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973791f2609b4b55aa986e9fd9baceec'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.126908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973791f2609b4b55aa986e9fd9baceec'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.140601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973791f2609b4b55aa986e9fd9baceec'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.142298 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:31:37.150740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973791f2609b4b55aa986e9fd9baceec'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.151639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973791f2609b4b55aa986e9fd9baceec'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.165463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1074/2141] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:31:37 2022] DELETE /identity/v3/users/973791f2609b4b55aa986e9fd9baceec => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:31:37.335656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:37.348433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee097356-3868-45e0-aa2b-cbdd1d5de921 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:37.406899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:37.407520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:37.407795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:37.408107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] PATH_INFO: `/v3/projects/cc21860aa70345c79cfe597d5a42fb51` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:37.409213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc21860aa70345c79cfe597d5a42fb51)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:37.411157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:37.456547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc21860aa70345c79cfe597d5a42fb51', 'request_id': 'req-e5dde65b-a599-4fba-931f-c6a0fb1228e6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.463510 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc21860aa70345c79cfe597d5a42fb51', 'request_id': 'req-e5dde65b-a599-4fba-931f-c6a0fb1228e6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.469170 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc21860aa70345c79cfe597d5a42fb51', 'request_id': 'req-e5dde65b-a599-4fba-931f-c6a0fb1228e6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.481155 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] The token cache is being invalidate because project cc21860aa70345c79cfe597d5a42fb51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:31:37.481443 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.482070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.483561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1068/2142] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:31:37 2022] DELETE /identity/v3/projects/cc21860aa70345c79cfe597d5a42fb51 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:31:37.487617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:37.489030 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:37.502085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fda17-71c5-482b-bc00-456f51d1d59d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:37.558696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:37.559308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:37.559580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:37.559846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] PATH_INFO: `/v3/users/39fa012345254c56b8abc88fd079022f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:37.560959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39fa012345254c56b8abc88fd079022f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:37.562879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:37.608687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39fa012345254c56b8abc88fd079022f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.609316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39fa012345254c56b8abc88fd079022f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.622154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39fa012345254c56b8abc88fd079022f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.629319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39fa012345254c56b8abc88fd079022f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.629932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39fa012345254c56b8abc88fd079022f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.642657 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1075/2143] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:31:37 2022] DELETE /identity/v3/users/39fa012345254c56b8abc88fd079022f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:31:37.816032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:31:37.817446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:31:37.832068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dde65b-a599-4fba-931f-c6a0fb1228e6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:31:37.837835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:31:37.892268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:31:37.892885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:31:37.893154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:31:37.893418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] PATH_INFO: `/v3/projects/2baaa5333cee4891a7d766231743d38e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:31:37.894587 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2baaa5333cee4891a7d766231743d38e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:31:37.896475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:31:37.941910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2baaa5333cee4891a7d766231743d38e', 'request_id': 'req-b52d6131-5a80-497f-8443-81c165ce7ef3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.948906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2baaa5333cee4891a7d766231743d38e', 'request_id': 'req-b52d6131-5a80-497f-8443-81c165ce7ef3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.954363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2baaa5333cee4891a7d766231743d38e', 'request_id': 'req-b52d6131-5a80-497f-8443-81c165ce7ef3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.967070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] The token cache is being invalidate because project 2baaa5333cee4891a7d766231743d38e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:31:37.967356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.968024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b52d6131-5a80-497f-8443-81c165ce7ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:31:37.969504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1069/2144] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:31:37 2022] DELETE /identity/v3/projects/2baaa5333cee4891a7d766231743d38e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:31:47.639333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=59 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:31:47.963686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:32:06.140347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:06.142415 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:06.212735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a6114-85ff-43b1-a78f-d1b5aeddf44b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:06.279163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:06.279791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:06.280413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:06.280413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:06.282707 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:06.350259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:06.351713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:06.361777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1076/2145] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:32:10.046235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e9ca-fd79-4711-9e63-c406a645de19 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:10.046769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e9ca-fd79-4711-9e63-c406a645de19 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:10.047219 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e9ca-fd79-4711-9e63-c406a645de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:10.078976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ef57e9ca-fd79-4711-9e63-c406a645de19 None None] MFA Rules not processed for user `eb7c6da9fd3c4b72a687daa923d07918`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:10.080530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef57e9ca-fd79-4711-9e63-c406a645de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:10.139410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1070/2146] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:32:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:10.201705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:10.203418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:10.217268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021d61-93aa-4998-96b3-517d7b4c4c05 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:10.222425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:10.223166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:10.223574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:10.223935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] PATH_INFO: `/v3/users/aaedb0e1920a42eb97e8dc2f3ec76905` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:10.225297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaedb0e1920a42eb97e8dc2f3ec76905)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:10.227391 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:10.288084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaedb0e1920a42eb97e8dc2f3ec76905'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.289129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaedb0e1920a42eb97e8dc2f3ec76905'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.300672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:32:10.303210 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaedb0e1920a42eb97e8dc2f3ec76905'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.312853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaedb0e1920a42eb97e8dc2f3ec76905'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.313486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-714ee896-0361-4f0d-bfe0-cfb5b53b9de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaedb0e1920a42eb97e8dc2f3ec76905'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.327333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1077/2147] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:32:10 2022] DELETE /identity/v3/users/aaedb0e1920a42eb97e8dc2f3ec76905 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:32:10.506681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e9ca-fd79-4711-9e63-c406a645de19 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:10.522875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e9ca-fd79-4711-9e63-c406a645de19 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:10.584408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:10.585034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:10.585307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:10.585573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] PATH_INFO: `/v3/projects/895b2d6d988f4ab7a0d9dcf9e130af42` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:10.587112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=895b2d6d988f4ab7a0d9dcf9e130af42)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:10.589041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:10.638812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895b2d6d988f4ab7a0d9dcf9e130af42', 'request_id': 'req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.647747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895b2d6d988f4ab7a0d9dcf9e130af42', 'request_id': 'req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.656680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895b2d6d988f4ab7a0d9dcf9e130af42', 'request_id': 'req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.673167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] The token cache is being invalidate because project 895b2d6d988f4ab7a0d9dcf9e130af42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:32:10.673453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.674072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe41d22-47f5-4770-a7c8-e577a4df7e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:32:10.675663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1071/2148] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:32:10 2022] DELETE /identity/v3/projects/895b2d6d988f4ab7a0d9dcf9e130af42 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:32:10.680807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc23d-5317-478e-a425-7315216c0c01 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:10.681219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc23d-5317-478e-a425-7315216c0c01 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:10.681404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc23d-5317-478e-a425-7315216c0c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:10.701495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f3ddc23d-5317-478e-a425-7315216c0c01 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:10.702714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f3ddc23d-5317-478e-a425-7315216c0c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:10.765365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1078/2149] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:10.776268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491e674-a40d-491e-8c7d-057889f9a6ce None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:10.776603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491e674-a40d-491e-8c7d-057889f9a6ce None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:10.776800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491e674-a40d-491e-8c7d-057889f9a6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:10.800140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c491e674-a40d-491e-8c7d-057889f9a6ce None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:10.801385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c491e674-a40d-491e-8c7d-057889f9a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:10.810243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1072/2150] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:10.814384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddc23d-5317-478e-a425-7315216c0c01 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:10.825519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddc23d-5317-478e-a425-7315216c0c01 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:10.830441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e38668-3002-4140-8316-501843384021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:10.830989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e38668-3002-4140-8316-501843384021 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:10.831266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e38668-3002-4140-8316-501843384021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:10.831533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e38668-3002-4140-8316-501843384021 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:10.832180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e38668-3002-4140-8316-501843384021 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:32:10.832481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e38668-3002-4140-8316-501843384021 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:10.834144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e38668-3002-4140-8316-501843384021 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:10.847814 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1079/2151] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:32:10.852305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491e674-a40d-491e-8c7d-057889f9a6ce None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:10.862179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491e674-a40d-491e-8c7d-057889f9a6ce None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:10.871328 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ef00-843b-4e87-be80-ef0f72352df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:10.872021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ef00-843b-4e87-be80-ef0f72352df4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:10.872300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ef00-843b-4e87-be80-ef0f72352df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:10.872569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ef00-843b-4e87-be80-ef0f72352df4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:10.873201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08ef00-843b-4e87-be80-ef0f72352df4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:10.875201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08ef00-843b-4e87-be80-ef0f72352df4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:10.908935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1073/2152] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:32:10 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:32:10.913063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e38668-3002-4140-8316-501843384021 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:10.924449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e38668-3002-4140-8316-501843384021 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:10.982295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021811a9-32bf-4dae-838d-f6788a52554d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:10.982954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021811a9-32bf-4dae-838d-f6788a52554d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:10.983228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021811a9-32bf-4dae-838d-f6788a52554d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:10.983501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021811a9-32bf-4dae-838d-f6788a52554d admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:10.984129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021811a9-32bf-4dae-838d-f6788a52554d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:10.986034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021811a9-32bf-4dae-838d-f6788a52554d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:11.008649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1080/2153] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:32:10 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:32:11.012901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ef00-843b-4e87-be80-ef0f72352df4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:11.024483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ef00-843b-4e87-be80-ef0f72352df4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:11.029607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f175b60-9223-4a5b-94cb-f7e66b5f4d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:11.030491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f175b60-9223-4a5b-94cb-f7e66b5f4d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:11.030772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f175b60-9223-4a5b-94cb-f7e66b5f4d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:11.031111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f175b60-9223-4a5b-94cb-f7e66b5f4d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:11.031841 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f175b60-9223-4a5b-94cb-f7e66b5f4d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:11.041894 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f175b60-9223-4a5b-94cb-f7e66b5f4d8d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:11.044679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:32:11.058708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1074/2154] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:32:11.059982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021811a9-32bf-4dae-838d-f6788a52554d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:11.071089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021811a9-32bf-4dae-838d-f6788a52554d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:11.076665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7f921-b3dd-4f70-abc1-3dce1ab49a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:11.077282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7f921-b3dd-4f70-abc1-3dce1ab49a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:11.077555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7f921-b3dd-4f70-abc1-3dce1ab49a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:11.077820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7f921-b3dd-4f70-abc1-3dce1ab49a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:11.078495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7f921-b3dd-4f70-abc1-3dce1ab49a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:11.080093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7f921-b3dd-4f70-abc1-3dce1ab49a4c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:11.095170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1081/2155] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:32:11.099322 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f175b60-9223-4a5b-94cb-f7e66b5f4d8d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:11.111823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f175b60-9223-4a5b-94cb-f7e66b5f4d8d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:11.117480 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:11.118150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:11.118685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:11.118958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] PATH_INFO: `/v3/projects/6a41305c4efa44edb98d1ca076ccbdff/users/e5efdc755c6a43f9b8cf570ac38b25eb/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:11.136287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a41305c4efa44edb98d1ca076ccbdff, user_id=e5efdc755c6a43f9b8cf570ac38b25eb, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:11.138709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:11.154599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1075/2156] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:32:11 2022] PUT /identity/v3/projects/6a41305c4efa44edb98d1ca076ccbdff/users/e5efdc755c6a43f9b8cf570ac38b25eb/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:32:11.160835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388973b2-c462-4f0c-bc58-bb8b75465f8f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:11.161297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388973b2-c462-4f0c-bc58-bb8b75465f8f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:11.161822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388973b2-c462-4f0c-bc58-bb8b75465f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:11.186188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-388973b2-c462-4f0c-bc58-bb8b75465f8f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:11.187547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-388973b2-c462-4f0c-bc58-bb8b75465f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:11.235314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:32:11.256516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1082/2157] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:11.267342 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:11.269233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:11.341482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9cb38-eec2-4e38-ab3b-f06e4b2e41c6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:11.345206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:11.345807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:11.346081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:11.346373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:11.360657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:11.361774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:11.372675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1076/2158] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:32:16.564753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87253418-1085-45fa-bc86-c4d0fdafa5f9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:16.565271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87253418-1085-45fa-bc86-c4d0fdafa5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:16.565422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87253418-1085-45fa-bc86-c4d0fdafa5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:16.614182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-87253418-1085-45fa-bc86-c4d0fdafa5f9 None None] MFA Rules not processed for user `e5efdc755c6a43f9b8cf570ac38b25eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:16.616796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-87253418-1085-45fa-bc86-c4d0fdafa5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:16.693259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1083/2159] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:32:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:16.708215 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:16.709831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:16.788486 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797f47c-5463-4d6b-8017-eceb3f4c5ae7 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:16.794420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:16.795010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:16.795286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:16.795549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:16.803134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:16.804711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:16.813698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1077/2160] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:32:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:32:18.420959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41999a-b1c8-4a8f-a6a2-fedc440f1c55 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:18.421503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41999a-b1c8-4a8f-a6a2-fedc440f1c55 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:18.421503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41999a-b1c8-4a8f-a6a2-fedc440f1c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:18.446511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6f41999a-b1c8-4a8f-a6a2-fedc440f1c55 None None] MFA Rules not processed for user `e5efdc755c6a43f9b8cf570ac38b25eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:18.447702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6f41999a-b1c8-4a8f-a6a2-fedc440f1c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:18.455739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1084/2161] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:32:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:18.789488 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:18.790980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:18.803094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebf6c2-1da5-4ed4-9a4d-dbe731819991 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:18.806957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c499f2c-e53f-43f7-b03d-995971d33b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:18.807634 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c499f2c-e53f-43f7-b03d-995971d33b04 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:18.808027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c499f2c-e53f-43f7-b03d-995971d33b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:18.808295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c499f2c-e53f-43f7-b03d-995971d33b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:18.810150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6c499f2c-e53f-43f7-b03d-995971d33b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:18.819888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c499f2c-e53f-43f7-b03d-995971d33b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:18.822470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c499f2c-e53f-43f7-b03d-995971d33b04 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:18.830859 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1078/2162] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:32:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:32:22.051037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca8ef-2ece-4682-b43b-88b5af0fe6a4 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:22.051583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca8ef-2ece-4682-b43b-88b5af0fe6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:22.051583 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca8ef-2ece-4682-b43b-88b5af0fe6a4 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:22.052908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1085/2163] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:32:22.061387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83f805-1f67-444b-9e50-13793780d09c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:22.061876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83f805-1f67-444b-9e50-13793780d09c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:22.061973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83f805-1f67-444b-9e50-13793780d09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:22.085476 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5b83f805-1f67-444b-9e50-13793780d09c None None] MFA Rules not processed for user `bd7883e3f6ab4f58969f51ffd439b1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:22.087960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b83f805-1f67-444b-9e50-13793780d09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:22.100298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1079/2164] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:32:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:22.123681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ca8ef-2ece-4682-b43b-88b5af0fe6a4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:22.138437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ca8ef-2ece-4682-b43b-88b5af0fe6a4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:22.144125 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4749b-ead3-4820-8ddb-4bd85b9b3404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:22.145537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4749b-ead3-4820-8ddb-4bd85b9b3404 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:22.146250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4749b-ead3-4820-8ddb-4bd85b9b3404 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:22.146757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4749b-ead3-4820-8ddb-4bd85b9b3404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:22.157909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4749b-ead3-4820-8ddb-4bd85b9b3404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:22.160126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4749b-ead3-4820-8ddb-4bd85b9b3404 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:22.172916 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1086/2165] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:32:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:32:22.330931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea739fc-1311-4b5f-abb4-de49be8cb999 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:22.331239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea739fc-1311-4b5f-abb4-de49be8cb999 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:22.331519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea739fc-1311-4b5f-abb4-de49be8cb999 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:22.332863 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1080/2166] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:32:22.338419 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b1cc-c0fb-4df4-bec0-e0b23874ebde None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:22.338776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b1cc-c0fb-4df4-bec0-e0b23874ebde None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:22.338971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b1cc-c0fb-4df4-bec0-e0b23874ebde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:22.359698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7827b1cc-c0fb-4df4-bec0-e0b23874ebde None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:22.360905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7827b1cc-c0fb-4df4-bec0-e0b23874ebde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:22.371061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1087/2167] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:32:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:22.381564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea739fc-1311-4b5f-abb4-de49be8cb999 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:32:22.393541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea739fc-1311-4b5f-abb4-de49be8cb999 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:32:22.397217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcdb29-e754-465a-b55a-6c9556baf004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:32:22.397939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcdb29-e754-465a-b55a-6c9556baf004 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:22.398317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcdb29-e754-465a-b55a-6c9556baf004 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:22.398975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcdb29-e754-465a-b55a-6c9556baf004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:22.411216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcdb29-e754-465a-b55a-6c9556baf004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:32:22.413168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcdb29-e754-465a-b55a-6c9556baf004 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:32:22.431963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1081/2168] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:32:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:32:32.354599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=51 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:32:32.426264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:32:38.508064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4de8a-e453-4085-afe4-743bcac7587f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:32:38.508597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4de8a-e453-4085-afe4-743bcac7587f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:32:38.508597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4de8a-e453-4085-afe4-743bcac7587f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:32:38.527822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2ab4de8a-e453-4085-afe4-743bcac7587f None None] MFA Rules not processed for user `6b4b2700c20946ea93a1d20d7dea929f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:32:38.529012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2ab4de8a-e453-4085-afe4-743bcac7587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:32:38.584265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1088/2169] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:32:48.577732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=25 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:33:24.021061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ad532-0383-4f33-9078-f5ec204daaf5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:33:24.021590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ad532-0383-4f33-9078-f5ec204daaf5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:33:24.021590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ad532-0383-4f33-9078-f5ec204daaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:33:24.050855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c67ad532-0383-4f33-9078-f5ec204daaf5 None None] MFA Rules not processed for user `a5b7190649694b9882ab34c0f3114a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:33:24.052042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c67ad532-0383-4f33-9078-f5ec204daaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:33:24.121201 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1082/2170] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:33:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:33:34.119441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=37 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:33:42.974457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4de8a-e453-4085-afe4-743bcac7587f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:33:43.012066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4de8a-e453-4085-afe4-743bcac7587f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:33:43.015685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:33:43.016272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:33:43.016546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:33:43.017596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:33:43.019373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:33:43.089251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:33:43.090733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:33:43.100394 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1089/2171] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:33:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:33:51.914198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ad532-0383-4f33-9078-f5ec204daaf5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:33:51.915565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c67ad532-0383-4f33-9078-f5ec204daaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:33:52.005647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ad532-0383-4f33-9078-f5ec204daaf5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:33:52.010654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:33:52.011254 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:33:52.011532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:33:52.011796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:33:52.013508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:33:52.025262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:33:52.026856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:33:52.036553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1083/2172] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:33:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:33:53.096067 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=45 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:02.032297 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=41 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:35.725722 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:35.735027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886e164-e93a-43e5-8bbe-3c2379ac67c2 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:35.739321 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:35.740771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:35.741162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:35.741431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:35.743257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:35.782351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:35.783908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:35.809699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1090/2173] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:34:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:34:39.407350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:39.417860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18febd-79ba-48a9-9524-ef7fb79d631b service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:39.421615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312346a-15a8-41d6-ab87-169ca7d2df2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:39.422211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312346a-15a8-41d6-ab87-169ca7d2df2d service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:39.422524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312346a-15a8-41d6-ab87-169ca7d2df2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:39.422792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312346a-15a8-41d6-ab87-169ca7d2df2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:39.424493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b312346a-15a8-41d6-ab87-169ca7d2df2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:39.438872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312346a-15a8-41d6-ab87-169ca7d2df2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:39.440509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312346a-15a8-41d6-ab87-169ca7d2df2d service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:39.452474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1084/2174] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:34:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:34:40.033548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:40.042875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373481eb-49a4-4948-b03f-5c23e1eceda9 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:40.046468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:40.047090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:40.047368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:40.047638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:40.049333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:40.084550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:40.086019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:40.106397 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1091/2175] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:34:40.762053 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0552fc-9c7f-404e-8f32-90272ec51308 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:40.762642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0552fc-9c7f-404e-8f32-90272ec51308 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:40.762806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0552fc-9c7f-404e-8f32-90272ec51308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:40.794752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4a0552fc-9c7f-404e-8f32-90272ec51308 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:34:40.795980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4a0552fc-9c7f-404e-8f32-90272ec51308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:40.859057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1085/2176] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:34:40.907197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:40.908602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:40.980566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f682a-5f59-40f5-87f7-592de952dfe0 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:40.986719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f7a48-bb9c-43c6-a36a-755b0d3b4c7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:40.987333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f7a48-bb9c-43c6-a36a-755b0d3b4c7b service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:40.987610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f7a48-bb9c-43c6-a36a-755b0d3b4c7b service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:40.987878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f7a48-bb9c-43c6-a36a-755b0d3b4c7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:40.996995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f7a48-bb9c-43c6-a36a-755b0d3b4c7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:40.998699 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f7a48-bb9c-43c6-a36a-755b0d3b4c7b service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:41.007770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1092/2177] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:34:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:34:44.271956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0552fc-9c7f-404e-8f32-90272ec51308 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:44.273954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4a0552fc-9c7f-404e-8f32-90272ec51308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:44.304546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0552fc-9c7f-404e-8f32-90272ec51308 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:44.309606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:44.310344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:44.310709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:44.311056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] PATH_INFO: `/v3/users/eb7c6da9fd3c4b72a687daa923d07918` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:44.329199 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb7c6da9fd3c4b72a687daa923d07918)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:44.331291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:44.391956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb7c6da9fd3c4b72a687daa923d07918'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.392648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb7c6da9fd3c4b72a687daa923d07918'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.404117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb7c6da9fd3c4b72a687daa923d07918'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.421664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb7c6da9fd3c4b72a687daa923d07918'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.422619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfba96e6-4c01-4db4-86d7-83ebdbd19dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb7c6da9fd3c4b72a687daa923d07918'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.437355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1086/2178] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:34:44 2022] DELETE /identity/v3/users/eb7c6da9fd3c4b72a687daa923d07918 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:34:44.670068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f7a48-bb9c-43c6-a36a-755b0d3b4c7b service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:44.683264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f7a48-bb9c-43c6-a36a-755b0d3b4c7b service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:44.752693 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:44.753374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:44.753656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:44.753930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] PATH_INFO: `/v3/projects/9ad3932c3fcc4d42b77b26f889ac694f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:44.767554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ad3932c3fcc4d42b77b26f889ac694f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:44.769568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:44.774390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:44.825861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ad3932c3fcc4d42b77b26f889ac694f', 'request_id': 'req-4a01f2b7-93d3-431a-86b3-adda053f7544'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.835181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ad3932c3fcc4d42b77b26f889ac694f', 'request_id': 'req-4a01f2b7-93d3-431a-86b3-adda053f7544'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.844270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ad3932c3fcc4d42b77b26f889ac694f', 'request_id': 'req-4a01f2b7-93d3-431a-86b3-adda053f7544'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.871641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] The token cache is being invalidate because project 9ad3932c3fcc4d42b77b26f889ac694f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:34:44.871969 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.878124 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a01f2b7-93d3-431a-86b3-adda053f7544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:34:44.881872 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1093/2179] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:34:44 2022] DELETE /identity/v3/projects/9ad3932c3fcc4d42b77b26f889ac694f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:34:44.888063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe3446-d743-4721-a634-b7468ccf478a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:44.888572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe3446-d743-4721-a634-b7468ccf478a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:44.888836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe3446-d743-4721-a634-b7468ccf478a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:44.976598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7afe3446-d743-4721-a634-b7468ccf478a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:34:44.978132 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7afe3446-d743-4721-a634-b7468ccf478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:44.988482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:45.064970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1087/2180] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:34:45.074634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4821d0-4ab8-4ed4-9e17-15a815a6076b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.074844 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4821d0-4ab8-4ed4-9e17-15a815a6076b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.075015 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4821d0-4ab8-4ed4-9e17-15a815a6076b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.107638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2d4821d0-4ab8-4ed4-9e17-15a815a6076b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:34:45.108882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2d4821d0-4ab8-4ed4-9e17-15a815a6076b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:45.119285 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1094/2181] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:34:45.127240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe3446-d743-4721-a634-b7468ccf478a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:45.140601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe3446-d743-4721-a634-b7468ccf478a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:45.145103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:45.145689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.146818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.147097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.147752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:34:45.148051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:45.152781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:45.179493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4821d0-4ab8-4ed4-9e17-15a815a6076b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:45.199767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1088/2182] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:34:45.203336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4821d0-4ab8-4ed4-9e17-15a815a6076b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:45.217717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e7048-fe32-4a0a-9a1f-14c0c7b73d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:45.218606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e7048-fe32-4a0a-9a1f-14c0c7b73d31 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.218887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e7048-fe32-4a0a-9a1f-14c0c7b73d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.219144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e7048-fe32-4a0a-9a1f-14c0c7b73d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.219778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e7048-fe32-4a0a-9a1f-14c0c7b73d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:45.221695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e7048-fe32-4a0a-9a1f-14c0c7b73d31 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:45.293429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1095/2183] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:34:45 2022] POST /identity/v3/projects => generated 355 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:34:45.298291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:45.312032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ab550-812a-4210-aaf7-7ba5083f814f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:45.425315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae3201-c84e-472b-a876-3db35e2f9808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:45.426124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae3201-c84e-472b-a876-3db35e2f9808 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.426599 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae3201-c84e-472b-a876-3db35e2f9808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.427656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae3201-c84e-472b-a876-3db35e2f9808 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.428613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae3201-c84e-472b-a876-3db35e2f9808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:45.430856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae3201-c84e-472b-a876-3db35e2f9808 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:45.464235 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1089/2184] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:34:45 2022] POST /identity/v3/users => generated 278 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:34:45.470649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e7048-fe32-4a0a-9a1f-14c0c7b73d31 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:45.484242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e7048-fe32-4a0a-9a1f-14c0c7b73d31 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:45.494262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35867b96-760c-46db-ade6-0429047cb5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:45.495435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35867b96-760c-46db-ade6-0429047cb5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.495815 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35867b96-760c-46db-ade6-0429047cb5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.496086 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35867b96-760c-46db-ade6-0429047cb5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.496726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35867b96-760c-46db-ade6-0429047cb5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:45.498316 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35867b96-760c-46db-ade6-0429047cb5d8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:45.512387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1096/2185] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:34:45.516116 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae3201-c84e-472b-a876-3db35e2f9808 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:45.528045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae3201-c84e-472b-a876-3db35e2f9808 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:45.532911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676a375-8911-4de0-98dc-7084a2ac88c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:45.533503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676a375-8911-4de0-98dc-7084a2ac88c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.533775 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676a375-8911-4de0-98dc-7084a2ac88c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.534040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676a375-8911-4de0-98dc-7084a2ac88c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.534713 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676a375-8911-4de0-98dc-7084a2ac88c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:45.537679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676a375-8911-4de0-98dc-7084a2ac88c8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:45.547908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1090/2186] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:34:45.552292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35867b96-760c-46db-ade6-0429047cb5d8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:45.562876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35867b96-760c-46db-ade6-0429047cb5d8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:45.569628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd1b6c-6f68-49c6-b5b2-b96546041896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:45.570313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1b6c-6f68-49c6-b5b2-b96546041896 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.571425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1b6c-6f68-49c6-b5b2-b96546041896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.571705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1b6c-6f68-49c6-b5b2-b96546041896 admin admin] PATH_INFO: `/v3/projects/3aaf9e43800b494e920ce745c0967f0f/users/6c41d2cd7f3f4ed1b767a3ccf0d4a191/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.588479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd1b6c-6f68-49c6-b5b2-b96546041896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aaf9e43800b494e920ce745c0967f0f, user_id=6c41d2cd7f3f4ed1b767a3ccf0d4a191, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:45.591159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd1b6c-6f68-49c6-b5b2-b96546041896 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:45.609741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1097/2187] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:34:45 2022] PUT /identity/v3/projects/3aaf9e43800b494e920ce745c0967f0f/users/6c41d2cd7f3f4ed1b767a3ccf0d4a191/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:34:45.620573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1629-9268-4048-8fd9-2dbedec4b181 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.620904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1629-9268-4048-8fd9-2dbedec4b181 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.621182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1629-9268-4048-8fd9-2dbedec4b181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.655096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-873b1629-9268-4048-8fd9-2dbedec4b181 None None] MFA Rules not processed for user `6c41d2cd7f3f4ed1b767a3ccf0d4a191`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:34:45.656538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-873b1629-9268-4048-8fd9-2dbedec4b181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:45.737584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1091/2188] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:34:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:34:45.742045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd1b6c-6f68-49c6-b5b2-b96546041896 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:45.835553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd1b6c-6f68-49c6-b5b2-b96546041896 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:45.840668 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b5992-b944-4bb1-b7a7-5e503e0a4724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:45.841275 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5992-b944-4bb1-b7a7-5e503e0a4724 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.841552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5992-b944-4bb1-b7a7-5e503e0a4724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.841944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5992-b944-4bb1-b7a7-5e503e0a4724 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.843783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b5992-b944-4bb1-b7a7-5e503e0a4724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:45.847200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b5992-b944-4bb1-b7a7-5e503e0a4724 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:45.887617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1098/2189] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:34:45 2022] POST /identity/v3/projects => generated 355 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:34:45.888306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b1629-9268-4048-8fd9-2dbedec4b181 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:45.900581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b1629-9268-4048-8fd9-2dbedec4b181 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:45.905456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:45.997163 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad099a-bf8c-44a7-b475-5a6fe54a1e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:45.997769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad099a-bf8c-44a7-b475-5a6fe54a1e9b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:45.998041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad099a-bf8c-44a7-b475-5a6fe54a1e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:45.998307 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad099a-bf8c-44a7-b475-5a6fe54a1e9b admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:45.998968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad099a-bf8c-44a7-b475-5a6fe54a1e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:46.000867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad099a-bf8c-44a7-b475-5a6fe54a1e9b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:46.026380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1092/2190] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:34:45 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:34:46.030238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b5992-b944-4bb1-b7a7-5e503e0a4724 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:46.048035 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:46.054117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b5992-b944-4bb1-b7a7-5e503e0a4724 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:46.062079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26e818-95e2-4c10-8445-dac7e644cd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:46.062842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26e818-95e2-4c10-8445-dac7e644cd37 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:46.063703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26e818-95e2-4c10-8445-dac7e644cd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:46.064127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26e818-95e2-4c10-8445-dac7e644cd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:46.065242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26e818-95e2-4c10-8445-dac7e644cd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:46.067213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26e818-95e2-4c10-8445-dac7e644cd37 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:46.104152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1099/2191] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:34:46.108920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad099a-bf8c-44a7-b475-5a6fe54a1e9b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:46.121917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad099a-bf8c-44a7-b475-5a6fe54a1e9b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:46.127597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32e491-89de-4f8e-908b-ac067da595ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:46.128273 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32e491-89de-4f8e-908b-ac067da595ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:46.128763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32e491-89de-4f8e-908b-ac067da595ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:46.128885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32e491-89de-4f8e-908b-ac067da595ec admin admin] PATH_INFO: `/v3/projects/e5f51322c0714e6ea874c27e9ad86a0f/users/7cda894181c74accac45dac137329cdf/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:46.145877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32e491-89de-4f8e-908b-ac067da595ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f51322c0714e6ea874c27e9ad86a0f, user_id=7cda894181c74accac45dac137329cdf, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:46.148953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32e491-89de-4f8e-908b-ac067da595ec admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:46.169151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1093/2192] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:34:46 2022] PUT /identity/v3/projects/e5f51322c0714e6ea874c27e9ad86a0f/users/7cda894181c74accac45dac137329cdf/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:34:46.170610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26e818-95e2-4c10-8445-dac7e644cd37 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:46.182173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26e818-95e2-4c10-8445-dac7e644cd37 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:46.245443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503e68a-b87c-4f9b-974d-6b89bb22fb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:46.246095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503e68a-b87c-4f9b-974d-6b89bb22fb86 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:46.246458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503e68a-b87c-4f9b-974d-6b89bb22fb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:46.246794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503e68a-b87c-4f9b-974d-6b89bb22fb86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:46.247622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503e68a-b87c-4f9b-974d-6b89bb22fb86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:46.249323 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503e68a-b87c-4f9b-974d-6b89bb22fb86 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:46.261252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1100/2193] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:34:46.265280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32e491-89de-4f8e-908b-ac067da595ec admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:46.276467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32e491-89de-4f8e-908b-ac067da595ec admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:46.281482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:46.282142 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:46.282483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:46.282753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] PATH_INFO: `/v3/projects/e5f51322c0714e6ea874c27e9ad86a0f/users/7cda894181c74accac45dac137329cdf/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:46.284699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f51322c0714e6ea874c27e9ad86a0f, user_id=7cda894181c74accac45dac137329cdf, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:46.287530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:46.304536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1094/2194] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:34:46 2022] PUT /identity/v3/projects/e5f51322c0714e6ea874c27e9ad86a0f/users/7cda894181c74accac45dac137329cdf/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:34:46.316184 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cdc29-b7d2-4c9d-b7d1-817ad871b141 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:46.316477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cdc29-b7d2-4c9d-b7d1-817ad871b141 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:46.316886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cdc29-b7d2-4c9d-b7d1-817ad871b141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:46.374460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b63cdc29-b7d2-4c9d-b7d1-817ad871b141 None None] MFA Rules not processed for user `7cda894181c74accac45dac137329cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:34:46.375746 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b63cdc29-b7d2-4c9d-b7d1-817ad871b141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:46.460569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1101/2195] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:34:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:34:46.475063 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:46.477821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:46.589453 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c48e6c-d3ca-4bbe-9590-432bb043a1bd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:46.601420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:46.602052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:46.602343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:46.602646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:46.615125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:46.616995 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:46.631315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1095/2196] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:34:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:34:46.735705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cdc29-b7d2-4c9d-b7d1-817ad871b141 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:46.748681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cdc29-b7d2-4c9d-b7d1-817ad871b141 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:46.759116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ae507-e97d-4943-9bfc-35f35914fdf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:46.759993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ae507-e97d-4943-9bfc-35f35914fdf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:46.760443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ae507-e97d-4943-9bfc-35f35914fdf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:46.760878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ae507-e97d-4943-9bfc-35f35914fdf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:46.844690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ae507-e97d-4943-9bfc-35f35914fdf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:46.847059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ae507-e97d-4943-9bfc-35f35914fdf8 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:46.858965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1102/2197] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:34:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:34:47.819868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:47.821846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:47.881735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:47.907509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989620ab-701c-4dbc-aa51-b129b9687c8c service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:47.911814 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:47.912433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:47.912703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:47.912978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:47.914897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:48.004721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:48.007221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:48.024667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1096/2198] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:34:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:34:54.882726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ae507-e97d-4943-9bfc-35f35914fdf8 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:54.892619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ae507-e97d-4943-9bfc-35f35914fdf8 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:54.896686 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafe0cb-33af-4b43-9363-1be630386c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:54.897357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafe0cb-33af-4b43-9363-1be630386c8e service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:54.897637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafe0cb-33af-4b43-9363-1be630386c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:54.897906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafe0cb-33af-4b43-9363-1be630386c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:54.899952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cfafe0cb-33af-4b43-9363-1be630386c8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:54.910546 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:54.978444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafe0cb-33af-4b43-9363-1be630386c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:54.980814 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafe0cb-33af-4b43-9363-1be630386c8e service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:54.995987 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1103/2199] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:34:58.020292 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=33 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:34:58.078248 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:34:58.080168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:58.099839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c019f-11d2-4b09-a573-f9ece464ebcd service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:34:58.108905 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d2a9a-0f9d-4b79-bcde-a10e6c194d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:34:58.109474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2a9a-0f9d-4b79-bcde-a10e6c194d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:34:58.109755 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2a9a-0f9d-4b79-bcde-a10e6c194d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:34:58.110028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2a9a-0f9d-4b79-bcde-a10e6c194d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:34:58.111980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d98d2a9a-0f9d-4b79-bcde-a10e6c194d25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:34:58.129008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d2a9a-0f9d-4b79-bcde-a10e6c194d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:34:58.130361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d2a9a-0f9d-4b79-bcde-a10e6c194d25 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:34:58.153029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1097/2200] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:00.114980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a03711-ceea-433d-88eb-5f083cad1a57 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:00.115531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a03711-ceea-433d-88eb-5f083cad1a57 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:00.115531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a03711-ceea-433d-88eb-5f083cad1a57 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:00.117493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1104/2201] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:35:00.127907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c672db-6175-4700-8f6a-6caeb89ddba7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:00.129028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c672db-6175-4700-8f6a-6caeb89ddba7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:00.129311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c672db-6175-4700-8f6a-6caeb89ddba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:00.202684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-94c672db-6175-4700-8f6a-6caeb89ddba7 None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:00.204064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94c672db-6175-4700-8f6a-6caeb89ddba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:00.357877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1098/2202] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:00.372529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a03711-ceea-433d-88eb-5f083cad1a57 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:00.375893 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-43a03711-ceea-433d-88eb-5f083cad1a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:00.501894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a03711-ceea-433d-88eb-5f083cad1a57 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:00.511001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c7cfb-3357-4c95-bf07-89cab25eab8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ac934ed0122470cb489d6b324c664f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:00.512568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c7cfb-3357-4c95-bf07-89cab25eab8b service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:00.513038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c7cfb-3357-4c95-bf07-89cab25eab8b service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:00.513552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c7cfb-3357-4c95-bf07-89cab25eab8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:00.531635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c7cfb-3357-4c95-bf07-89cab25eab8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:00.533289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c7cfb-3357-4c95-bf07-89cab25eab8b service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:00.557713 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1105/2203] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:00.675537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf1ce-a290-47a0-b9e6-0f0d272a1d1c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:00.676159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf1ce-a290-47a0-b9e6-0f0d272a1d1c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:00.677234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf1ce-a290-47a0-b9e6-0f0d272a1d1c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:00.678897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1099/2204] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:35:00.684058 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a8bb-f883-4c47-8817-663c6a83be2a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:00.684499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a8bb-f883-4c47-8817-663c6a83be2a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:00.684869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a8bb-f883-4c47-8817-663c6a83be2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:00.755522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1721a8bb-f883-4c47-8817-663c6a83be2a None None] MFA Rules not processed for user `bd7883e3f6ab4f58969f51ffd439b1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:00.757019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1721a8bb-f883-4c47-8817-663c6a83be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:00.778618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1106/2205] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:01.046169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf1ce-a290-47a0-b9e6-0f0d272a1d1c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:01.087365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf1ce-a290-47a0-b9e6-0f0d272a1d1c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:01.097625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8198f54-20e6-4035-8856-772122374410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:01.103734 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8198f54-20e6-4035-8856-772122374410 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:01.104439 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8198f54-20e6-4035-8856-772122374410 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:01.105598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8198f54-20e6-4035-8856-772122374410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:01.132857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8198f54-20e6-4035-8856-772122374410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:01.134317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8198f54-20e6-4035-8856-772122374410 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:01.150493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1100/2206] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:01.400243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625c5e8-c65e-40b5-8279-792bffc4a347 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:01.401265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625c5e8-c65e-40b5-8279-792bffc4a347 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:01.401750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625c5e8-c65e-40b5-8279-792bffc4a347 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:01.403484 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1107/2207] 10.222.0.16 () {54 vars in 921 bytes} [Mon Oct 17 10:35:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 10:35:01.419979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f52b47-87ab-454b-8ee5-b44927f44925 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:01.420260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f52b47-87ab-454b-8ee5-b44927f44925 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:01.420754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f52b47-87ab-454b-8ee5-b44927f44925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:01.459350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d9f52b47-87ab-454b-8ee5-b44927f44925 None None] MFA Rules not processed for user `9d52624718de4457adc44e9965cd7a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:01.463798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9f52b47-87ab-454b-8ee5-b44927f44925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:01.494474 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1101/2208] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Oct 17 10:35:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:01.596549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625c5e8-c65e-40b5-8279-792bffc4a347 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:01.618172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625c5e8-c65e-40b5-8279-792bffc4a347 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:01.631878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:01.632862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:01.633403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:01.633873 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:01.649173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:01.651178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:01.668633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1108/2209] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:03.175081 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f52b47-87ab-454b-8ee5-b44927f44925 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:03.179182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9f52b47-87ab-454b-8ee5-b44927f44925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:03.204350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f52b47-87ab-454b-8ee5-b44927f44925 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:03.225846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:03.228767 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:03.229125 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:03.229505 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] PATH_INFO: `/v3/users/6b4b2700c20946ea93a1d20d7dea929f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:03.302387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4b2700c20946ea93a1d20d7dea929f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:03.304923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:03.383513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4b2700c20946ea93a1d20d7dea929f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.384528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4b2700c20946ea93a1d20d7dea929f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.398255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4b2700c20946ea93a1d20d7dea929f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.423817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4b2700c20946ea93a1d20d7dea929f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.425319 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4b2700c20946ea93a1d20d7dea929f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.438879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1102/2210] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:03 2022] DELETE /identity/v3/users/6b4b2700c20946ea93a1d20d7dea929f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:03.477364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:03.480456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:03.506876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40e8e-ab96-4ea1-93e6-48ab81667032 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:03.584052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:03.584913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:03.585291 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:03.585565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] PATH_INFO: `/v3/users/75018aeecd6843c0a75bc95e8757fd8a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:03.586762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=75018aeecd6843c0a75bc95e8757fd8a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:03.588694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:03.607781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:03.656848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:03.657615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75018aeecd6843c0a75bc95e8757fd8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.658552 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:03.658813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75018aeecd6843c0a75bc95e8757fd8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.677048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75018aeecd6843c0a75bc95e8757fd8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.679919 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e61f50-87c0-42ca-b098-370f3253bb66 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:03.687178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75018aeecd6843c0a75bc95e8757fd8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.689103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0be1d9a-38f1-4e79-88ca-c2d8b5085b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75018aeecd6843c0a75bc95e8757fd8a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.700463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:03.701021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:03.701298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:03.701871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] PATH_INFO: `/v3/projects/41a236cfbaf14211a6a3674f3c2f3a67` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:03.704672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:03.715039 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1109/2211] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:03 2022] DELETE /identity/v3/users/75018aeecd6843c0a75bc95e8757fd8a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:03.721605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a236cfbaf14211a6a3674f3c2f3a67)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:03.724667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:03.787248 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a236cfbaf14211a6a3674f3c2f3a67', 'request_id': 'req-4105a915-c225-43dd-bad6-9c77372f23b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.794528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a236cfbaf14211a6a3674f3c2f3a67', 'request_id': 'req-4105a915-c225-43dd-bad6-9c77372f23b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.802052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a236cfbaf14211a6a3674f3c2f3a67', 'request_id': 'req-4105a915-c225-43dd-bad6-9c77372f23b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.816034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] The token cache is being invalidate because project 41a236cfbaf14211a6a3674f3c2f3a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:03.816320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.817246 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4105a915-c225-43dd-bad6-9c77372f23b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:03.819212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1103/2212] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:03 2022] DELETE /identity/v3/projects/41a236cfbaf14211a6a3674f3c2f3a67 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:03.824592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8c462-97b6-4492-bd80-bcce25243375 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:03.825051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8c462-97b6-4492-bd80-bcce25243375 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:03.825446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8c462-97b6-4492-bd80-bcce25243375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:03.849361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-79c8c462-97b6-4492-bd80-bcce25243375 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:03.850780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-79c8c462-97b6-4492-bd80-bcce25243375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:03.927427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1110/2213] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:03.935561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9651bd7-0096-4073-bc28-4ce1a593eb88 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:03.935842 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9651bd7-0096-4073-bc28-4ce1a593eb88 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:03.936137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9651bd7-0096-4073-bc28-4ce1a593eb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:03.966417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f9651bd7-0096-4073-bc28-4ce1a593eb88 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:03.968313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f9651bd7-0096-4073-bc28-4ce1a593eb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:03.978127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1104/2214] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:03.982700 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8c462-97b6-4492-bd80-bcce25243375 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:03.994633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8c462-97b6-4492-bd80-bcce25243375 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:03.999624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.000236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.000511 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.000780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.001444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:04.001901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.003721 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.018457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1111/2215] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:35:04.022615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9651bd7-0096-4073-bc28-4ce1a593eb88 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.032230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9651bd7-0096-4073-bc28-4ce1a593eb88 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.039373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.040197 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.040541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.040869 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.041576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.043645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.086674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1105/2216] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:04 2022] POST /identity/v3/projects => generated 405 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:04.091727 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.105563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40a77c-af27-4b0c-997a-ab3e9a882141 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.185230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.187111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.196749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952516c-8192-4a2c-8b42-853bf4c08c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.197379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952516c-8192-4a2c-8b42-853bf4c08c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.197655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952516c-8192-4a2c-8b42-853bf4c08c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.198001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952516c-8192-4a2c-8b42-853bf4c08c6a admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.198751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952516c-8192-4a2c-8b42-853bf4c08c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.201801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6705c3-f9e1-428e-ab21-0ab90ca2590c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.202853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952516c-8192-4a2c-8b42-853bf4c08c6a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.226450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.227195 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.227571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.227941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] PATH_INFO: `/v3/projects/8ca835fdc95849cd8db1882ebb693cc3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.229238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca835fdc95849cd8db1882ebb693cc3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.233317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1112/2217] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:04 2022] POST /identity/v3/users => generated 303 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:04.233418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.234145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952516c-8192-4a2c-8b42-853bf4c08c6a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.246810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952516c-8192-4a2c-8b42-853bf4c08c6a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.257562 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ce11-d25f-4a9a-b31c-be376c52c2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.258179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ce11-d25f-4a9a-b31c-be376c52c2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.258696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ce11-d25f-4a9a-b31c-be376c52c2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.258800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ce11-d25f-4a9a-b31c-be376c52c2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.259424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ce11-d25f-4a9a-b31c-be376c52c2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.261223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ce11-d25f-4a9a-b31c-be376c52c2c8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.268986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:04.279238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1113/2218] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:04.280693 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ce11-d25f-4a9a-b31c-be376c52c2c8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.291700 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ce11-d25f-4a9a-b31c-be376c52c2c8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.315131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca835fdc95849cd8db1882ebb693cc3', 'request_id': 'req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.330853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca835fdc95849cd8db1882ebb693cc3', 'request_id': 'req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.339156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca835fdc95849cd8db1882ebb693cc3', 'request_id': 'req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.363601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] The token cache is being invalidate because project 8ca835fdc95849cd8db1882ebb693cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:04.364435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.365728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.367871 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1106/2219] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:04 2022] DELETE /identity/v3/projects/8ca835fdc95849cd8db1882ebb693cc3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:04.368908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.372798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.386597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.387205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.387619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.387895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.388917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.391343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.410514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1114/2220] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:04.410514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.412705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.426817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:04.430641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863807-35d8-40e6-89ff-c2aba0e4c539 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.447571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.448247 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.448519 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.448789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] PATH_INFO: `/v3/users/e71ce5b5697f43f786e949a940f10c1e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.450059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71ce5b5697f43f786e949a940f10c1e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.454129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.454241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac26ec-11cc-4175-a6cf-da710ef7d0e8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.458892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.459502 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.459786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.460058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.461826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.474251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.475899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.484700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1107/2221] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:04.495981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.497560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.513242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205f801-9300-4265-aa59-a02760344473 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.518915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7013d1-3990-478e-84c7-7b1652ec53b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.519542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7013d1-3990-478e-84c7-7b1652ec53b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.519762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7013d1-3990-478e-84c7-7b1652ec53b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.520700 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7013d1-3990-478e-84c7-7b1652ec53b1 admin admin] PATH_INFO: `/v3/projects/8dc2455ccf06479c8e9a68b11bdc31fb/users/0f401df91f304f369640b42df6279c8c/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.530859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71ce5b5697f43f786e949a940f10c1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.532761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71ce5b5697f43f786e949a940f10c1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.546560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71ce5b5697f43f786e949a940f10c1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.551972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7013d1-3990-478e-84c7-7b1652ec53b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc2455ccf06479c8e9a68b11bdc31fb, user_id=0f401df91f304f369640b42df6279c8c, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.554739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71ce5b5697f43f786e949a940f10c1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.555404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7013d1-3990-478e-84c7-7b1652ec53b1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.555587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71ce5b5697f43f786e949a940f10c1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.572578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1115/2222] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:04 2022] DELETE /identity/v3/users/e71ce5b5697f43f786e949a940f10c1e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:04.572578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.572878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1108/2223] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:04 2022] PUT /identity/v3/projects/8dc2455ccf06479c8e9a68b11bdc31fb/users/0f401df91f304f369640b42df6279c8c/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:04.575091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.587938 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f10325-e964-433b-b9df-9034b39d8046 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.588332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f10325-e964-433b-b9df-9034b39d8046 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.588852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f10325-e964-433b-b9df-9034b39d8046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.624665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f1f10325-e964-433b-b9df-9034b39d8046 None None] MFA Rules not processed for user `0f401df91f304f369640b42df6279c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:04.626074 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f1f10325-e964-433b-b9df-9034b39d8046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.671114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36518efa-b54a-4c83-9faa-ec6c5563d19f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.677996 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.678903 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.679767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.680192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] PATH_INFO: `/v3/users/6c41d2cd7f3f4ed1b767a3ccf0d4a191` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.681548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c41d2cd7f3f4ed1b767a3ccf0d4a191)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.684172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.699995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:04.713303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1109/2224] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:04 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:04.718030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f10325-e964-433b-b9df-9034b39d8046 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.719741 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f1f10325-e964-433b-b9df-9034b39d8046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.737981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f10325-e964-433b-b9df-9034b39d8046 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.744109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.744838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.745227 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.745624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.746499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.748602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.750685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c41d2cd7f3f4ed1b767a3ccf0d4a191'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.751881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c41d2cd7f3f4ed1b767a3ccf0d4a191'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.773656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c41d2cd7f3f4ed1b767a3ccf0d4a191'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.783705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c41d2cd7f3f4ed1b767a3ccf0d4a191'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.784807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c41d2cd7f3f4ed1b767a3ccf0d4a191'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.789300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1110/2225] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:04 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:04.793300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.797851 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.804165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1116/2226] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:04 2022] DELETE /identity/v3/users/6c41d2cd7f3f4ed1b767a3ccf0d4a191 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:04.805287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.808235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.887470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19d7b0-1033-454c-b46a-32c18759cb75 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.893375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.894025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.894491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.894769 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] PATH_INFO: `/v3/projects/fdd2f2b7ddfd420c8f133852dbeea70c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.896346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd2f2b7ddfd420c8f133852dbeea70c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.896748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da2618-a816-4c52-b8a8-76f8b8f41fc3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:04.898260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.902191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2948af-583a-46ea-8bda-c1ad0d2f39c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:04.902896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2948af-583a-46ea-8bda-c1ad0d2f39c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.903231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2948af-583a-46ea-8bda-c1ad0d2f39c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.903562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2948af-583a-46ea-8bda-c1ad0d2f39c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.904259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2948af-583a-46ea-8bda-c1ad0d2f39c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:04.907139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2948af-583a-46ea-8bda-c1ad0d2f39c4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:04.930158 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1111/2227] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:04 2022] POST /identity/v3/users => generated 309 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:04.931965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1389b-693d-4878-bfa1-61e11b76f81d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:04.932243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1389b-693d-4878-bfa1-61e11b76f81d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:04.932583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1389b-693d-4878-bfa1-61e11b76f81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:04.953818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd2f2b7ddfd420c8f133852dbeea70c', 'request_id': 'req-b63ab710-7908-4dec-9237-bf3483d98ddd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.955239 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cad1389b-693d-4878-bfa1-61e11b76f81d None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:04.956495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cad1389b-693d-4878-bfa1-61e11b76f81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:04.960970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd2f2b7ddfd420c8f133852dbeea70c', 'request_id': 'req-b63ab710-7908-4dec-9237-bf3483d98ddd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.963286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:04.969927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd2f2b7ddfd420c8f133852dbeea70c', 'request_id': 'req-b63ab710-7908-4dec-9237-bf3483d98ddd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.982413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] The token cache is being invalidate because project fdd2f2b7ddfd420c8f133852dbeea70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:04.982788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.983522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:04.985819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1117/2228] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:04 2022] DELETE /identity/v3/projects/fdd2f2b7ddfd420c8f133852dbeea70c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:04.986518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:04.988005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.018296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:05.053581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1112/2229] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:05.066673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1389b-693d-4878-bfa1-61e11b76f81d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.069384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cad1389b-693d-4878-bfa1-61e11b76f81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.102700 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ab710-7908-4dec-9237-bf3483d98ddd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.112430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6d45f-e3ae-4702-80ed-fd6fabf09524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.113336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6d45f-e3ae-4702-80ed-fd6fabf09524 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.114208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6d45f-e3ae-4702-80ed-fd6fabf09524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.114752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6d45f-e3ae-4702-80ed-fd6fabf09524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.115721 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6d45f-e3ae-4702-80ed-fd6fabf09524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.117478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6d45f-e3ae-4702-80ed-fd6fabf09524 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.136804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1118/2230] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:05.138667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6d45f-e3ae-4702-80ed-fd6fabf09524 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.154104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6d45f-e3ae-4702-80ed-fd6fabf09524 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.161239 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4354ece-ce59-4efa-8b11-6264575f55ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.162597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4354ece-ce59-4efa-8b11-6264575f55ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.163119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4354ece-ce59-4efa-8b11-6264575f55ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.163503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4354ece-ce59-4efa-8b11-6264575f55ce admin admin] PATH_INFO: `/v3/projects/2a319257359542bb83f7d30929926dac/users/cded6eeeef9d4ac493b00011e893c548/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.165457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1389b-693d-4878-bfa1-61e11b76f81d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.169353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74e2dd-c792-469d-b52e-acd629dd5c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.169979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e2dd-c792-469d-b52e-acd629dd5c67 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.170257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e2dd-c792-469d-b52e-acd629dd5c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.170581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e2dd-c792-469d-b52e-acd629dd5c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.183940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4354ece-ce59-4efa-8b11-6264575f55ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a319257359542bb83f7d30929926dac, user_id=cded6eeeef9d4ac493b00011e893c548, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.184301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74e2dd-c792-469d-b52e-acd629dd5c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.187012 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74e2dd-c792-469d-b52e-acd629dd5c67 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.187694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4354ece-ce59-4efa-8b11-6264575f55ce admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.208138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1113/2231] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:05.218628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74e2dd-c792-469d-b52e-acd629dd5c67 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.226177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1119/2232] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:05 2022] PUT /identity/v3/projects/2a319257359542bb83f7d30929926dac/users/cded6eeeef9d4ac493b00011e893c548/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:05.234369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74e2dd-c792-469d-b52e-acd629dd5c67 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.308604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e3183-80c0-4cce-b5c9-24a120298b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.309210 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3183-80c0-4cce-b5c9-24a120298b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.309631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3183-80c0-4cce-b5c9-24a120298b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.309902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3183-80c0-4cce-b5c9-24a120298b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.310603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e3183-80c0-4cce-b5c9-24a120298b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.312252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e3183-80c0-4cce-b5c9-24a120298b55 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.323521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1114/2233] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:05.328325 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4354ece-ce59-4efa-8b11-6264575f55ce admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.338553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4354ece-ce59-4efa-8b11-6264575f55ce admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.343178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884b00c-bbef-4b4c-9cbb-6f2d1d04f583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.343823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884b00c-bbef-4b4c-9cbb-6f2d1d04f583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.344104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884b00c-bbef-4b4c-9cbb-6f2d1d04f583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.344371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884b00c-bbef-4b4c-9cbb-6f2d1d04f583 admin admin] PATH_INFO: `/v3/projects/2a319257359542bb83f7d30929926dac/users/cded6eeeef9d4ac493b00011e893c548/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.346263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884b00c-bbef-4b4c-9cbb-6f2d1d04f583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a319257359542bb83f7d30929926dac, user_id=cded6eeeef9d4ac493b00011e893c548, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.349475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884b00c-bbef-4b4c-9cbb-6f2d1d04f583 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.360974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1120/2234] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:05 2022] PUT /identity/v3/projects/2a319257359542bb83f7d30929926dac/users/cded6eeeef9d4ac493b00011e893c548/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:05.371813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33909e75-45ce-4f2f-b1a5-070cc305220d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.374904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33909e75-45ce-4f2f-b1a5-070cc305220d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.374904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33909e75-45ce-4f2f-b1a5-070cc305220d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.403450 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-33909e75-45ce-4f2f-b1a5-070cc305220d None None] MFA Rules not processed for user `cded6eeeef9d4ac493b00011e893c548`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:05.404760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-33909e75-45ce-4f2f-b1a5-070cc305220d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.456729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:05.487683 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f73381-84ad-4c7f-b42e-89a58a6d2c6a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.488139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f73381-84ad-4c7f-b42e-89a58a6d2c6a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.488623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f73381-84ad-4c7f-b42e-89a58a6d2c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.494034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1115/2235] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:05 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:05.502358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33909e75-45ce-4f2f-b1a5-070cc305220d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.503082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-33909e75-45ce-4f2f-b1a5-070cc305220d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.559776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c1f73381-84ad-4c7f-b42e-89a58a6d2c6a None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:05.561003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c1f73381-84ad-4c7f-b42e-89a58a6d2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.580935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33909e75-45ce-4f2f-b1a5-070cc305220d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.585277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.585908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.586181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.586481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.596078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.597493 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.608458 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1116/2236] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:05 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:05.619115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.622113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.663673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1121/2237] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:35:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:05.689680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f73381-84ad-4c7f-b42e-89a58a6d2c6a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.691271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c1f73381-84ad-4c7f-b42e-89a58a6d2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.711384 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c9f46-f028-468a-8d97-77ba6b8dd51a service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.716147 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.716785 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.717064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.717338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] PATH_INFO: `/v3/projects/3aaf9e43800b494e920ce745c0967f0f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.718642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aaf9e43800b494e920ce745c0967f0f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.720573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.790790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aaf9e43800b494e920ce745c0967f0f', 'request_id': 'req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.795004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:05.797675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aaf9e43800b494e920ce745c0967f0f', 'request_id': 'req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.805870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aaf9e43800b494e920ce745c0967f0f', 'request_id': 'req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.821831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] The token cache is being invalidate because project 3aaf9e43800b494e920ce745c0967f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:05.822128 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.822860 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.824427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1117/2238] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:05 2022] DELETE /identity/v3/projects/3aaf9e43800b494e920ce745c0967f0f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:05.825252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.827538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.838561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:05.839908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f73381-84ad-4c7f-b42e-89a58a6d2c6a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.895943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa257f7-3e90-4623-9aa8-1fc91d2c45ad admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:05.900779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.901425 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.901705 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.901970 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] PATH_INFO: `/v3/users/0f401df91f304f369640b42df6279c8c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.903240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f401df91f304f369640b42df6279c8c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.905143 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.906479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:05.907090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.907371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.907669 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:05.909475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.954761 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f401df91f304f369640b42df6279c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.955433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f401df91f304f369640b42df6279c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.963335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f401df91f304f369640b42df6279c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.971617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:05.973781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:05.976622 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f401df91f304f369640b42df6279c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.977304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f672f012-e735-4143-9d57-64240abc1ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f401df91f304f369640b42df6279c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:05.982197 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1122/2239] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:35:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 10:35:05.989984 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:05.991364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:05.991930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1118/2240] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:05 2022] DELETE /identity/v3/users/0f401df91f304f369640b42df6279c8c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:05.996308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cd58-6700-4c31-bd8e-b54266dd896b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:05.996605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cd58-6700-4c31-bd8e-b54266dd896b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:05.996884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cd58-6700-4c31-bd8e-b54266dd896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:06.010631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200fe5-5fae-4d61-ae56-596c058c31c9 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:06.018070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-aa46cd58-6700-4c31-bd8e-b54266dd896b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:06.019399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aa46cd58-6700-4c31-bd8e-b54266dd896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:06.086624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:06.087734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:06.088112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:06.088481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] PATH_INFO: `/v3/users/7cda894181c74accac45dac137329cdf` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:06.088855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1119/2241] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:06.090072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cda894181c74accac45dac137329cdf)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:06.094437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:06.101329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cd58-6700-4c31-bd8e-b54266dd896b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:06.103967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aa46cd58-6700-4c31-bd8e-b54266dd896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:06.114863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:06.147968 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:06.187025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cd58-6700-4c31-bd8e-b54266dd896b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:06.192340 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:06.193426 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:06.194006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:06.194484 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:06.196733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cda894181c74accac45dac137329cdf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.199313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cda894181c74accac45dac137329cdf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.207208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:06.210013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:06.225622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cda894181c74accac45dac137329cdf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.237255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1120/2242] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:06.250402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cda894181c74accac45dac137329cdf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.251303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cda894181c74accac45dac137329cdf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.272011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1123/2243] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:05 2022] DELETE /identity/v3/users/7cda894181c74accac45dac137329cdf => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:06.640934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:06.642662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:06.673186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cf21e-f116-4d51-ac8a-de5f1b823200 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:06.752604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:06.754195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:06.800129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:06.802037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:06.802337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:06.802714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] PATH_INFO: `/v3/projects/8dc2455ccf06479c8e9a68b11bdc31fb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:06.805935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc2455ccf06479c8e9a68b11bdc31fb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:06.810295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:06.856827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9f994-e8d8-4fce-b04a-9a416a84777a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:06.862640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:06.863428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:06.863949 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:06.864282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] PATH_INFO: `/v3/projects/e5f51322c0714e6ea874c27e9ad86a0f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:06.865697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f51322c0714e6ea874c27e9ad86a0f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:06.868156 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:06.906840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc2455ccf06479c8e9a68b11bdc31fb', 'request_id': 'req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.916941 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc2455ccf06479c8e9a68b11bdc31fb', 'request_id': 'req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.926120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc2455ccf06479c8e9a68b11bdc31fb', 'request_id': 'req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.947335 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f51322c0714e6ea874c27e9ad86a0f', 'request_id': 'req-ec20abcb-6f22-4618-ae87-80eb81062e50'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.954317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] The token cache is being invalidate because project 8dc2455ccf06479c8e9a68b11bdc31fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:06.954553 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f51322c0714e6ea874c27e9ad86a0f', 'request_id': 'req-ec20abcb-6f22-4618-ae87-80eb81062e50'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.954665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.956101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.962007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1121/2244] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:06 2022] DELETE /identity/v3/projects/8dc2455ccf06479c8e9a68b11bdc31fb => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:06.963470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:06.965154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:06.967907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f51322c0714e6ea874c27e9ad86a0f', 'request_id': 'req-ec20abcb-6f22-4618-ae87-80eb81062e50'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.983571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] The token cache is being invalidate because project e5f51322c0714e6ea874c27e9ad86a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:06.984070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.984593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec20abcb-6f22-4618-ae87-80eb81062e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:06.986414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1124/2245] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:06 2022] DELETE /identity/v3/projects/e5f51322c0714e6ea874c27e9ad86a0f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:06.993250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85987784-0555-4281-a015-98275b60d81c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:06.993540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85987784-0555-4281-a015-98275b60d81c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:06.993788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85987784-0555-4281-a015-98275b60d81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.033046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-85987784-0555-4281-a015-98275b60d81c None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:07.041276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-85987784-0555-4281-a015-98275b60d81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:07.077995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:07.086604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ecbb-2f2a-4168-97f2-7cad2bef0c30 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.093691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:07.151671 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1125/2246] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:07.159886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299463a5-58bc-43af-9b6a-f699aa4ffac2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.160207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299463a5-58bc-43af-9b6a-f699aa4ffac2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.160847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299463a5-58bc-43af-9b6a-f699aa4ffac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.162017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.162674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.162950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.163220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] PATH_INFO: `/v3/users/cded6eeeef9d4ac493b00011e893c548` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.164357 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cded6eeeef9d4ac493b00011e893c548)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.166293 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.181681 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-299463a5-58bc-43af-9b6a-f699aa4ffac2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:07.184574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-299463a5-58bc-43af-9b6a-f699aa4ffac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:07.191665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1126/2247] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:07.197254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299463a5-58bc-43af-9b6a-f699aa4ffac2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.207301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299463a5-58bc-43af-9b6a-f699aa4ffac2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.212889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.213550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.213851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.214122 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.214798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:07.215367 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.216961 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.220025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cded6eeeef9d4ac493b00011e893c548'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.220694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cded6eeeef9d4ac493b00011e893c548'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.228092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cded6eeeef9d4ac493b00011e893c548'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.233743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1127/2248] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:35:07.234716 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.236585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:07.245131 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cded6eeeef9d4ac493b00011e893c548'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.245807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cded6eeeef9d4ac493b00011e893c548'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.250033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12671ca8-63b9-41a2-81b3-6a466f15e1c5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.255412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93f02-d8ab-474c-bf1b-dcac38e0a010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.255998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93f02-d8ab-474c-bf1b-dcac38e0a010 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.256271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93f02-d8ab-474c-bf1b-dcac38e0a010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.256535 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93f02-d8ab-474c-bf1b-dcac38e0a010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.257154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c93f02-d8ab-474c-bf1b-dcac38e0a010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.259069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c93f02-d8ab-474c-bf1b-dcac38e0a010 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.259343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1122/2249] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:06 2022] DELETE /identity/v3/users/cded6eeeef9d4ac493b00011e893c548 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:07.287188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1128/2250] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:07.291185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.358038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111770a-3311-41a6-95a2-8467d003a5f5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.392343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:07.423987 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de417a50-9d41-471f-ac85-f5e8d28f0904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.424779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417a50-9d41-471f-ac85-f5e8d28f0904 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.425154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417a50-9d41-471f-ac85-f5e8d28f0904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.425483 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417a50-9d41-471f-ac85-f5e8d28f0904 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.426290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de417a50-9d41-471f-ac85-f5e8d28f0904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.429196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de417a50-9d41-471f-ac85-f5e8d28f0904 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.454140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1123/2251] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/users => generated 281 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:07.455623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93f02-d8ab-474c-bf1b-dcac38e0a010 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.466540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93f02-d8ab-474c-bf1b-dcac38e0a010 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.471150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944db7-5684-4bfe-8a95-1455b78b0e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.471748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944db7-5684-4bfe-8a95-1455b78b0e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.472023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944db7-5684-4bfe-8a95-1455b78b0e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.472293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944db7-5684-4bfe-8a95-1455b78b0e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.472935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944db7-5684-4bfe-8a95-1455b78b0e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.475416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944db7-5684-4bfe-8a95-1455b78b0e42 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.486102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1129/2252] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:07.490820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de417a50-9d41-471f-ac85-f5e8d28f0904 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.500730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de417a50-9d41-471f-ac85-f5e8d28f0904 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.506920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.507536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.507807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.508071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.508704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.510287 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.522901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1124/2253] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:07.525915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944db7-5684-4bfe-8a95-1455b78b0e42 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.536738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944db7-5684-4bfe-8a95-1455b78b0e42 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.542030 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d85019-5fd3-47eb-a0ad-64e932b02fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.542929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d85019-5fd3-47eb-a0ad-64e932b02fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.543288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d85019-5fd3-47eb-a0ad-64e932b02fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.543660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d85019-5fd3-47eb-a0ad-64e932b02fac admin admin] PATH_INFO: `/v3/projects/eb74f9685d2545ca978732738871962f/users/5f70317a67e442ed902e30c6b775234b/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.548007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.550143 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:07.561152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d85019-5fd3-47eb-a0ad-64e932b02fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb74f9685d2545ca978732738871962f, user_id=5f70317a67e442ed902e30c6b775234b, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.563752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d85019-5fd3-47eb-a0ad-64e932b02fac admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.565826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd715ea-936f-4bab-aad6-c3f63c04ba8e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.571969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.572604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.573783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.574103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] PATH_INFO: `/v3/projects/2a319257359542bb83f7d30929926dac` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.576272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a319257359542bb83f7d30929926dac)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.580934 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1130/2254] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:07 2022] PUT /identity/v3/projects/eb74f9685d2545ca978732738871962f/users/5f70317a67e442ed902e30c6b775234b/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:07.582080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.592772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6eb1-c236-4837-9752-ec739ab17cbf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.593158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6eb1-c236-4837-9752-ec739ab17cbf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.593525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6eb1-c236-4837-9752-ec739ab17cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.637117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c21e6eb1-c236-4837-9752-ec739ab17cbf None None] MFA Rules not processed for user `5f70317a67e442ed902e30c6b775234b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:07.638266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c21e6eb1-c236-4837-9752-ec739ab17cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:07.641558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a319257359542bb83f7d30929926dac', 'request_id': 'req-33572404-d5c0-49fe-8e36-4fe830823e3c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.659828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a319257359542bb83f7d30929926dac', 'request_id': 'req-33572404-d5c0-49fe-8e36-4fe830823e3c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.667664 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a319257359542bb83f7d30929926dac', 'request_id': 'req-33572404-d5c0-49fe-8e36-4fe830823e3c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.681466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] The token cache is being invalidate because project 2a319257359542bb83f7d30929926dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:07.681885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.682771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-33572404-d5c0-49fe-8e36-4fe830823e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:07.685048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1125/2255] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:07 2022] DELETE /identity/v3/projects/2a319257359542bb83f7d30929926dac => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:07.689808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e006-c625-4cc8-b3b5-c745f5f659b7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.690092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e006-c625-4cc8-b3b5-c745f5f659b7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.690424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e006-c625-4cc8-b3b5-c745f5f659b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.712679 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f346e006-c625-4cc8-b3b5-c745f5f659b7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:07.713895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f346e006-c625-4cc8-b3b5-c745f5f659b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:07.717098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1131/2256] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:07.721207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e6eb1-c236-4837-9752-ec739ab17cbf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.722802 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c21e6eb1-c236-4837-9752-ec739ab17cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:07.778524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:07.803588 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1126/2257] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:07.810419 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e6eb1-c236-4837-9752-ec739ab17cbf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.811115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b51f3-5f48-4ebe-83c4-a6a3303f30cb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.811374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b51f3-5f48-4ebe-83c4-a6a3303f30cb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.811643 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b51f3-5f48-4ebe-83c4-a6a3303f30cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.834268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8a1b51f3-5f48-4ebe-83c4-a6a3303f30cb None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:07.835596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8a1b51f3-5f48-4ebe-83c4-a6a3303f30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:07.837089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc648fee-0887-4df6-95c9-435369e3da4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.838026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc648fee-0887-4df6-95c9-435369e3da4e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.838544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc648fee-0887-4df6-95c9-435369e3da4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.839152 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc648fee-0887-4df6-95c9-435369e3da4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.839795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc648fee-0887-4df6-95c9-435369e3da4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.841811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc648fee-0887-4df6-95c9-435369e3da4e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.844024 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1127/2258] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:07.848283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b51f3-5f48-4ebe-83c4-a6a3303f30cb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.860171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b51f3-5f48-4ebe-83c4-a6a3303f30cb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.868766 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.869386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.869714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.869998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.870944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:07.871249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.873257 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.877878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1132/2259] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:07.882292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc648fee-0887-4df6-95c9-435369e3da4e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.893944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1128/2260] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:35:07.894878 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:07.905492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a185-7fec-4b93-9af7-6c68a807f859 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.920401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:07.976710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc648fee-0887-4df6-95c9-435369e3da4e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:07.981573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3da8f-6daf-4423-ba3d-9267faf2d342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.982309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3da8f-6daf-4423-ba3d-9267faf2d342 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.982723 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3da8f-6daf-4423-ba3d-9267faf2d342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.983094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3da8f-6daf-4423-ba3d-9267faf2d342 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.983829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3da8f-6daf-4423-ba3d-9267faf2d342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.985960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3da8f-6daf-4423-ba3d-9267faf2d342 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:07.990649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e883-cb44-4e19-836b-bf5bff6b2bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:07.991272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e883-cb44-4e19-836b-bf5bff6b2bed admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:07.991550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e883-cb44-4e19-836b-bf5bff6b2bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:07.991821 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e883-cb44-4e19-836b-bf5bff6b2bed admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:07.992457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536e883-cb44-4e19-836b-bf5bff6b2bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:07.994428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536e883-cb44-4e19-836b-bf5bff6b2bed admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.024139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1133/2261] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:08.027999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e883-cb44-4e19-836b-bf5bff6b2bed admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.030936 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1129/2262] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:07 2022] POST /identity/v3/projects => generated 405 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:08.035165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3da8f-6daf-4423-ba3d-9267faf2d342 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.042211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e883-cb44-4e19-836b-bf5bff6b2bed admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.109701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3da8f-6daf-4423-ba3d-9267faf2d342 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.113482 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418ce03-bd72-41b1-ac10-50a6cfb594b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.114869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418ce03-bd72-41b1-ac10-50a6cfb594b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.115368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418ce03-bd72-41b1-ac10-50a6cfb594b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.115628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a120dc-e9f0-4ee8-bea6-05a344d9e04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.115997 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418ce03-bd72-41b1-ac10-50a6cfb594b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.116264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a120dc-e9f0-4ee8-bea6-05a344d9e04d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.116536 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a120dc-e9f0-4ee8-bea6-05a344d9e04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.116803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a120dc-e9f0-4ee8-bea6-05a344d9e04d admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.117380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418ce03-bd72-41b1-ac10-50a6cfb594b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.117514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a120dc-e9f0-4ee8-bea6-05a344d9e04d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.119328 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418ce03-bd72-41b1-ac10-50a6cfb594b2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.119478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a120dc-e9f0-4ee8-bea6-05a344d9e04d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.139805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1134/2263] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:08.144337 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418ce03-bd72-41b1-ac10-50a6cfb594b2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.189460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418ce03-bd72-41b1-ac10-50a6cfb594b2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.195479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1bd4c-c2ea-4904-8550-d59999738153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.196209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1bd4c-c2ea-4904-8550-d59999738153 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.196574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1bd4c-c2ea-4904-8550-d59999738153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.197271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1bd4c-c2ea-4904-8550-d59999738153 admin admin] PATH_INFO: `/v3/projects/330bd2f7d64a4ae5b1585063e99dacd4/users/22fe6fd7aefb46d59c2b5efd952e4da5/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.197402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1130/2264] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:08 2022] POST /identity/v3/users => generated 303 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:08.198306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a120dc-e9f0-4ee8-bea6-05a344d9e04d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.209412 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a120dc-e9f0-4ee8-bea6-05a344d9e04d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.214285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92252fc0-057e-4e88-905b-ce5c370b7c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.214916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92252fc0-057e-4e88-905b-ce5c370b7c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.215190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92252fc0-057e-4e88-905b-ce5c370b7c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.215455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92252fc0-057e-4e88-905b-ce5c370b7c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.216127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92252fc0-057e-4e88-905b-ce5c370b7c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.217687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92252fc0-057e-4e88-905b-ce5c370b7c33 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.218983 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1bd4c-c2ea-4904-8550-d59999738153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=330bd2f7d64a4ae5b1585063e99dacd4, user_id=22fe6fd7aefb46d59c2b5efd952e4da5, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.223820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1bd4c-c2ea-4904-8550-d59999738153 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.244078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1131/2265] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:08.249632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92252fc0-057e-4e88-905b-ce5c370b7c33 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.265001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1135/2266] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:08 2022] PUT /identity/v3/projects/330bd2f7d64a4ae5b1585063e99dacd4/users/22fe6fd7aefb46d59c2b5efd952e4da5/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:08.268926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1bd4c-c2ea-4904-8550-d59999738153 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.270692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92252fc0-057e-4e88-905b-ce5c370b7c33 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.312911 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:08.371858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1bd4c-c2ea-4904-8550-d59999738153 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.377464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc754a74-7241-4231-90a3-91e0fbf4dc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.378085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc754a74-7241-4231-90a3-91e0fbf4dc00 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.378427 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc754a74-7241-4231-90a3-91e0fbf4dc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.378779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc754a74-7241-4231-90a3-91e0fbf4dc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.380366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc754a74-7241-4231-90a3-91e0fbf4dc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.382130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc754a74-7241-4231-90a3-91e0fbf4dc00 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.389126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901a96a-b8f3-46df-ab10-b82f571958d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.389739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901a96a-b8f3-46df-ab10-b82f571958d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.390015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901a96a-b8f3-46df-ab10-b82f571958d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.390377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901a96a-b8f3-46df-ab10-b82f571958d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.391403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901a96a-b8f3-46df-ab10-b82f571958d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.393097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901a96a-b8f3-46df-ab10-b82f571958d5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.399613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1136/2267] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:08.400287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc754a74-7241-4231-90a3-91e0fbf4dc00 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.412603 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1132/2268] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:08.413707 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901a96a-b8f3-46df-ab10-b82f571958d5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.414032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc754a74-7241-4231-90a3-91e0fbf4dc00 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.431491 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901a96a-b8f3-46df-ab10-b82f571958d5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.432358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903df05-6e04-401b-b8cd-15ce0ff46d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.434096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903df05-6e04-401b-b8cd-15ce0ff46d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.435371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903df05-6e04-401b-b8cd-15ce0ff46d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.435649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903df05-6e04-401b-b8cd-15ce0ff46d58 admin admin] PATH_INFO: `/v3/projects/330bd2f7d64a4ae5b1585063e99dacd4/users/22fe6fd7aefb46d59c2b5efd952e4da5/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.438299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903df05-6e04-401b-b8cd-15ce0ff46d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=330bd2f7d64a4ae5b1585063e99dacd4, user_id=22fe6fd7aefb46d59c2b5efd952e4da5, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.442548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903df05-6e04-401b-b8cd-15ce0ff46d58 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.444753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9b410-ad3c-447f-9def-b8085430c9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.445429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b410-ad3c-447f-9def-b8085430c9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.445709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b410-ad3c-447f-9def-b8085430c9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.445981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b410-ad3c-447f-9def-b8085430c9dc admin admin] PATH_INFO: `/v3/projects/9d4ade881513407eb839b7ac557acdf0/users/9f84851e9b994c50a1163a479f1a97e7/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.455441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1137/2269] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:08 2022] PUT /identity/v3/projects/330bd2f7d64a4ae5b1585063e99dacd4/users/22fe6fd7aefb46d59c2b5efd952e4da5/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:08.468464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dfe04-1118-43aa-9ce8-0b9bc0a7e4bf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.468804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dfe04-1118-43aa-9ce8-0b9bc0a7e4bf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.469063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dfe04-1118-43aa-9ce8-0b9bc0a7e4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.473523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9b410-ad3c-447f-9def-b8085430c9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d4ade881513407eb839b7ac557acdf0, user_id=9f84851e9b994c50a1163a479f1a97e7, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.476904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9b410-ad3c-447f-9def-b8085430c9dc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.481451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:08.497440 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1133/2270] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:08 2022] PUT /identity/v3/projects/9d4ade881513407eb839b7ac557acdf0/users/9f84851e9b994c50a1163a479f1a97e7/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:08.504876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36c4d0-33e5-4d08-aa63-632204405dde None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.505272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36c4d0-33e5-4d08-aa63-632204405dde None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.505644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36c4d0-33e5-4d08-aa63-632204405dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.526205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ee36c4d0-33e5-4d08-aa63-632204405dde None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:08.527314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-aa2dfe04-1118-43aa-9ce8-0b9bc0a7e4bf None None] MFA Rules not processed for user `22fe6fd7aefb46d59c2b5efd952e4da5`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:08.527726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee36c4d0-33e5-4d08-aa63-632204405dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:08.528634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aa2dfe04-1118-43aa-9ce8-0b9bc0a7e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:08.596708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1134/2271] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:08.603781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36c4d0-33e5-4d08-aa63-632204405dde None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.605471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee36c4d0-33e5-4d08-aa63-632204405dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:08.612233 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1138/2272] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:08.624606 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dfe04-1118-43aa-9ce8-0b9bc0a7e4bf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.626005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aa2dfe04-1118-43aa-9ce8-0b9bc0a7e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:08.675999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36c4d0-33e5-4d08-aa63-632204405dde None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.680499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9e0e9-4d79-452f-a1c0-b48f9e980827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.681159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9e0e9-4d79-452f-a1c0-b48f9e980827 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.681988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9e0e9-4d79-452f-a1c0-b48f9e980827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.681988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9e0e9-4d79-452f-a1c0-b48f9e980827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.696206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9e0e9-4d79-452f-a1c0-b48f9e980827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.697902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9e0e9-4d79-452f-a1c0-b48f9e980827 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.709920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1135/2273] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:08.713662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dfe04-1118-43aa-9ce8-0b9bc0a7e4bf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.717361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec82586-75b1-4fe1-9549-9b6826e20acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.717971 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82586-75b1-4fe1-9549-9b6826e20acb service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.718244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82586-75b1-4fe1-9549-9b6826e20acb service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.718543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82586-75b1-4fe1-9549-9b6826e20acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.729384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82586-75b1-4fe1-9549-9b6826e20acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.734721 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82586-75b1-4fe1-9549-9b6826e20acb service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.747592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1139/2274] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:08.849672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9e0e9-4d79-452f-a1c0-b48f9e980827 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:08.861673 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9e0e9-4d79-452f-a1c0-b48f9e980827 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:08.866469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169c24-5776-444f-bb86-2e59dd4f8b38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:08.867150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169c24-5776-444f-bb86-2e59dd4f8b38 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:08.867462 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169c24-5776-444f-bb86-2e59dd4f8b38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:08.867763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169c24-5776-444f-bb86-2e59dd4f8b38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:08.949548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a169c24-5776-444f-bb86-2e59dd4f8b38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:08.951837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a169c24-5776-444f-bb86-2e59dd4f8b38 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:08.966824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1136/2275] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:09.643314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613304e-6b71-4da2-9d90-afc967c67768 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:09.644024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613304e-6b71-4da2-9d90-afc967c67768 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:09.644024 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613304e-6b71-4da2-9d90-afc967c67768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:09.682074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a613304e-6b71-4da2-9d90-afc967c67768 None None] MFA Rules not processed for user `9f84851e9b994c50a1163a479f1a97e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:09.683387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a613304e-6b71-4da2-9d90-afc967c67768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:09.719250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:09.795702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1140/2276] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:09.800399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169c24-5776-444f-bb86-2e59dd4f8b38 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:09.812486 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169c24-5776-444f-bb86-2e59dd4f8b38 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:09.817884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476751b-698c-4391-b95b-86208d35315d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:09.818745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476751b-698c-4391-b95b-86208d35315d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:09.819192 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476751b-698c-4391-b95b-86208d35315d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:09.819619 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476751b-698c-4391-b95b-86208d35315d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:09.820424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476751b-698c-4391-b95b-86208d35315d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:09.835658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476751b-698c-4391-b95b-86208d35315d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:09.870380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1137/2277] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:09 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:09.873208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613304e-6b71-4da2-9d90-afc967c67768 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:09.884717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613304e-6b71-4da2-9d90-afc967c67768 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:09.944937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cee168-5a48-4862-9d19-b2d5684f2f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:09.945557 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee168-5a48-4862-9d19-b2d5684f2f9c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:09.945838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee168-5a48-4862-9d19-b2d5684f2f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:09.946110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee168-5a48-4862-9d19-b2d5684f2f9c admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:09.946794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cee168-5a48-4862-9d19-b2d5684f2f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:09.948795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cee168-5a48-4862-9d19-b2d5684f2f9c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:09.972253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1141/2278] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:09 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:09.976279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476751b-698c-4391-b95b-86208d35315d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:09.987279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476751b-698c-4391-b95b-86208d35315d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:09.992290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe37a1-1d8a-44ec-a1aa-557f9a81e5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:09.992893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe37a1-1d8a-44ec-a1aa-557f9a81e5af admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:09.993172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe37a1-1d8a-44ec-a1aa-557f9a81e5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:09.993445 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe37a1-1d8a-44ec-a1aa-557f9a81e5af admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:09.994096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe37a1-1d8a-44ec-a1aa-557f9a81e5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:09.995737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe37a1-1d8a-44ec-a1aa-557f9a81e5af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:10.009810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1138/2279] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:10.013252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cee168-5a48-4862-9d19-b2d5684f2f9c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:10.025102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cee168-5a48-4862-9d19-b2d5684f2f9c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:10.029770 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950783cc-e071-4d85-bf50-7f0eb9adb50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:10.030465 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950783cc-e071-4d85-bf50-7f0eb9adb50c admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:10.030745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950783cc-e071-4d85-bf50-7f0eb9adb50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:10.031018 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950783cc-e071-4d85-bf50-7f0eb9adb50c admin admin] PATH_INFO: `/v3/projects/be2d4ace471843028b3f49aeab8508f2/users/9c905a9778df412da3b234d312f3aa24/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:10.047314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950783cc-e071-4d85-bf50-7f0eb9adb50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2d4ace471843028b3f49aeab8508f2, user_id=9c905a9778df412da3b234d312f3aa24, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:10.050242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950783cc-e071-4d85-bf50-7f0eb9adb50c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:10.065515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1142/2280] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:10 2022] PUT /identity/v3/projects/be2d4ace471843028b3f49aeab8508f2/users/9c905a9778df412da3b234d312f3aa24/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:10.070244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe37a1-1d8a-44ec-a1aa-557f9a81e5af admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:10.075424 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:10.080724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe37a1-1d8a-44ec-a1aa-557f9a81e5af admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:10.157427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5dfdb-0815-4846-a073-2f8b8cfeb271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:10.158044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5dfdb-0815-4846-a073-2f8b8cfeb271 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:10.158422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5dfdb-0815-4846-a073-2f8b8cfeb271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:10.158732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5dfdb-0815-4846-a073-2f8b8cfeb271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:10.159396 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5dfdb-0815-4846-a073-2f8b8cfeb271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:10.161003 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5dfdb-0815-4846-a073-2f8b8cfeb271 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:10.172085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1139/2281] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:10.176709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950783cc-e071-4d85-bf50-7f0eb9adb50c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:10.198084 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950783cc-e071-4d85-bf50-7f0eb9adb50c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:10.205724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c235b5-d799-4309-9727-da3a00f56e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:10.206556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c235b5-d799-4309-9727-da3a00f56e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:10.206835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c235b5-d799-4309-9727-da3a00f56e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:10.207117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c235b5-d799-4309-9727-da3a00f56e9f admin admin] PATH_INFO: `/v3/projects/be2d4ace471843028b3f49aeab8508f2/users/9c905a9778df412da3b234d312f3aa24/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:10.209100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c235b5-d799-4309-9727-da3a00f56e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2d4ace471843028b3f49aeab8508f2, user_id=9c905a9778df412da3b234d312f3aa24, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:10.211880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c235b5-d799-4309-9727-da3a00f56e9f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:10.237909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1143/2282] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:10 2022] PUT /identity/v3/projects/be2d4ace471843028b3f49aeab8508f2/users/9c905a9778df412da3b234d312f3aa24/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:11.116617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6f92-cfa9-4a87-8208-94155ba77210 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:11.117591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6f92-cfa9-4a87-8208-94155ba77210 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:11.118092 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6f92-cfa9-4a87-8208-94155ba77210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:11.156118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a9fb6f92-cfa9-4a87-8208-94155ba77210 None None] MFA Rules not processed for user `9c905a9778df412da3b234d312f3aa24`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:11.157409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a9fb6f92-cfa9-4a87-8208-94155ba77210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:11.254355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1140/2283] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:12.078674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c235b5-d799-4309-9727-da3a00f56e9f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:12.177290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c235b5-d799-4309-9727-da3a00f56e9f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:12.183010 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:12.184779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:12.185398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:12.185823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] PATH_INFO: `/v3/users/5f70317a67e442ed902e30c6b775234b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:12.187198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f70317a67e442ed902e30c6b775234b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:12.189155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:12.232751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:12.306356 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f70317a67e442ed902e30c6b775234b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.307173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f70317a67e442ed902e30c6b775234b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.322591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f70317a67e442ed902e30c6b775234b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.333215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f70317a67e442ed902e30c6b775234b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.333999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f70317a67e442ed902e30c6b775234b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.349324 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1144/2284] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:12 2022] DELETE /identity/v3/users/5f70317a67e442ed902e30c6b775234b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:12.353676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47b3e-033b-4392-ae5a-daed0a8e4218 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:12.354185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47b3e-033b-4392-ae5a-daed0a8e4218 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:12.354437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47b3e-033b-4392-ae5a-daed0a8e4218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:12.374716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bfa47b3e-033b-4392-ae5a-daed0a8e4218 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:12.376565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bfa47b3e-033b-4392-ae5a-daed0a8e4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:12.515167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1141/2285] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:12.537757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:12.540263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:12.658687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccacffb-d04e-45a9-aa97-8440ec7d58c9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:12.668574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:12.669480 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:12.670302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:12.671268 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:12.686822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:12.688728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:12.705211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1145/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:12.718593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa47b3e-033b-4392-ae5a-daed0a8e4218 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:12.720394 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bfa47b3e-033b-4392-ae5a-daed0a8e4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:12.774353 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa47b3e-033b-4392-ae5a-daed0a8e4218 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:12.781004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:12.781660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:12.781942 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:12.782209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] PATH_INFO: `/v3/users/9f84851e9b994c50a1163a479f1a97e7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:12.784015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f84851e9b994c50a1163a479f1a97e7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:12.788658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:12.839875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:12.879171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f84851e9b994c50a1163a479f1a97e7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.880639 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f84851e9b994c50a1163a479f1a97e7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.899163 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f84851e9b994c50a1163a479f1a97e7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.927862 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f84851e9b994c50a1163a479f1a97e7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.928740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f84851e9b994c50a1163a479f1a97e7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:12.951837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1142/2287] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:12 2022] DELETE /identity/v3/users/9f84851e9b994c50a1163a479f1a97e7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:13.155771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:13.157098 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:13.173216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873e9d7-0f87-4d08-b7cb-eec0af827724 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:13.240314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:13.241845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:13.295529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:13.296733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:13.297475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:13.297897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] PATH_INFO: `/v3/projects/eb74f9685d2545ca978732738871962f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:13.299655 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb74f9685d2545ca978732738871962f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:13.308318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:13.377712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:13.389641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb74f9685d2545ca978732738871962f', 'request_id': 'req-e01e1928-c769-4280-a0df-16c09f52bf3b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.389993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c726c-7c73-4f1e-bf5a-4e1177aca215 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:13.400108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb74f9685d2545ca978732738871962f', 'request_id': 'req-e01e1928-c769-4280-a0df-16c09f52bf3b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.409750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb74f9685d2545ca978732738871962f', 'request_id': 'req-e01e1928-c769-4280-a0df-16c09f52bf3b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.439477 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] The token cache is being invalidate because project eb74f9685d2545ca978732738871962f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:13.439774 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.442710 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.446072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1146/2288] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:13 2022] DELETE /identity/v3/projects/eb74f9685d2545ca978732738871962f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:13.451801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:13.455538 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:13.548657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:13.549121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:13.550104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:13.550470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] PATH_INFO: `/v3/projects/9d4ade881513407eb839b7ac557acdf0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:13.553387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d4ade881513407eb839b7ac557acdf0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:13.555790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:13.588598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e1928-c769-4280-a0df-16c09f52bf3b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:13.604736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:13.606042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:13.606595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:13.607087 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] PATH_INFO: `/v3/users/22fe6fd7aefb46d59c2b5efd952e4da5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:13.610181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=22fe6fd7aefb46d59c2b5efd952e4da5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:13.616595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:13.645533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:13.649923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d4ade881513407eb839b7ac557acdf0', 'request_id': 'req-9122915d-48ba-4016-aff8-b356a31866ec'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.665030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d4ade881513407eb839b7ac557acdf0', 'request_id': 'req-9122915d-48ba-4016-aff8-b356a31866ec'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.672852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d4ade881513407eb839b7ac557acdf0', 'request_id': 'req-9122915d-48ba-4016-aff8-b356a31866ec'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.681463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22fe6fd7aefb46d59c2b5efd952e4da5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.686624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22fe6fd7aefb46d59c2b5efd952e4da5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.691373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] The token cache is being invalidate because project 9d4ade881513407eb839b7ac557acdf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:13.692319 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.693181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.695902 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1143/2289] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:13 2022] DELETE /identity/v3/projects/9d4ade881513407eb839b7ac557acdf0 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:13.700198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:13.706882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:13.717178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22fe6fd7aefb46d59c2b5efd952e4da5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.731571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22fe6fd7aefb46d59c2b5efd952e4da5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.732308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22fe6fd7aefb46d59c2b5efd952e4da5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.755778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1147/2290] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:13 2022] DELETE /identity/v3/users/22fe6fd7aefb46d59c2b5efd952e4da5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:13.799626 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122915d-48ba-4016-aff8-b356a31866ec admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:13.888047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:13.888882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:13.889295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:13.889568 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] PATH_INFO: `/v3/users/9c905a9778df412da3b234d312f3aa24` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:13.890777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c905a9778df412da3b234d312f3aa24)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:13.892722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:13.972300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c905a9778df412da3b234d312f3aa24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.973171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c905a9778df412da3b234d312f3aa24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:13.983472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c905a9778df412da3b234d312f3aa24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.018264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c905a9778df412da3b234d312f3aa24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.019051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c905a9778df412da3b234d312f3aa24'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.031055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:14.040583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1144/2291] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:13 2022] DELETE /identity/v3/users/9c905a9778df412da3b234d312f3aa24 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:14.205059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:14.208831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:14.238448 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1067c4-e354-4849-b8f8-a6348f22c2fc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:14.249348 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:14.250850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:14.342292 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:14.363079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:14.364398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.364738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.365077 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] PATH_INFO: `/v3/projects/330bd2f7d64a4ae5b1585063e99dacd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.366579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] RBAC: Authorizing `identity:delete_project(project_id=330bd2f7d64a4ae5b1585063e99dacd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:14.368968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:14.376291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60eceb1-eb22-490c-8cbc-f57e3bffe123 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:14.381988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:14.382660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.382934 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.383204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] PATH_INFO: `/v3/projects/be2d4ace471843028b3f49aeab8508f2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.384337 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2d4ace471843028b3f49aeab8508f2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:14.386489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:14.442782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '330bd2f7d64a4ae5b1585063e99dacd4', 'request_id': 'req-ff19582c-869a-4830-aea6-0b3f2940446f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.450772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '330bd2f7d64a4ae5b1585063e99dacd4', 'request_id': 'req-ff19582c-869a-4830-aea6-0b3f2940446f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.462547 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '330bd2f7d64a4ae5b1585063e99dacd4', 'request_id': 'req-ff19582c-869a-4830-aea6-0b3f2940446f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.468396 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2d4ace471843028b3f49aeab8508f2', 'request_id': 'req-e92a4346-27de-4c63-919f-08e2bf526584'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.477198 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2d4ace471843028b3f49aeab8508f2', 'request_id': 'req-e92a4346-27de-4c63-919f-08e2bf526584'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.479155 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] The token cache is being invalidate because project 330bd2f7d64a4ae5b1585063e99dacd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:14.480343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.484351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff19582c-869a-4830-aea6-0b3f2940446f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.488051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1148/2292] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:14 2022] DELETE /identity/v3/projects/330bd2f7d64a4ae5b1585063e99dacd4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:14.493338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75df1c-5890-40fd-b0d7-baa40d0316c2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.493868 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75df1c-5890-40fd-b0d7-baa40d0316c2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.494488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75df1c-5890-40fd-b0d7-baa40d0316c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.495279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2d4ace471843028b3f49aeab8508f2', 'request_id': 'req-e92a4346-27de-4c63-919f-08e2bf526584'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.514319 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] The token cache is being invalidate because project be2d4ace471843028b3f49aeab8508f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:14.514981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.517744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e92a4346-27de-4c63-919f-08e2bf526584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:14.525548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1145/2293] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:14 2022] DELETE /identity/v3/projects/be2d4ace471843028b3f49aeab8508f2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:14.528507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e15884-5fef-4bc7-8034-21099308923e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.529020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e15884-5fef-4bc7-8034-21099308923e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.529469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e15884-5fef-4bc7-8034-21099308923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.541891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fb75df1c-5890-40fd-b0d7-baa40d0316c2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:14.544731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb75df1c-5890-40fd-b0d7-baa40d0316c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:14.563322 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-44e15884-5fef-4bc7-8034-21099308923e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:14.564999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44e15884-5fef-4bc7-8034-21099308923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:14.655037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1149/2294] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:14.664253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411b70-66c5-4387-9709-249d6dbf87d6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.664736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411b70-66c5-4387-9709-249d6dbf87d6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.665336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411b70-66c5-4387-9709-249d6dbf87d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.695464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1146/2295] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:14.696808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fa411b70-66c5-4387-9709-249d6dbf87d6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:14.698013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fa411b70-66c5-4387-9709-249d6dbf87d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:14.703517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba502-fd88-4cd1-91b9-f84897858581 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.703888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba502-fd88-4cd1-91b9-f84897858581 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.704152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba502-fd88-4cd1-91b9-f84897858581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.710506 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1150/2296] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:14.714721 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa411b70-66c5-4387-9709-249d6dbf87d6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:14.724554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa411b70-66c5-4387-9709-249d6dbf87d6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:14.729552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:14.730598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.730957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.731321 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.732068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:14.732460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:14.734078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-160ba502-fd88-4cd1-91b9-f84897858581 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:14.734491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:14.735366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-160ba502-fd88-4cd1-91b9-f84897858581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:14.752346 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1151/2297] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:35:14.753565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:14.758518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1147/2298] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:14.759873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba502-fd88-4cd1-91b9-f84897858581 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:14.765966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f3631-f436-4298-920a-0132f100fdcb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:14.772158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:14.773083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.773560 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.774993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.775129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba502-fd88-4cd1-91b9-f84897858581 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:14.775976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:14.776973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:14.779234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:14.780120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c96db7-923f-46cb-ad7e-08c0789480cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:14.780848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c96db7-923f-46cb-ad7e-08c0789480cf admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.781191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c96db7-923f-46cb-ad7e-08c0789480cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.782027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c96db7-923f-46cb-ad7e-08c0789480cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.782442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c96db7-923f-46cb-ad7e-08c0789480cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:14.785834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c96db7-923f-46cb-ad7e-08c0789480cf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:14.797315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1152/2299] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:35:14.802847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:14.821222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380a303-8a5a-4cc6-8f1b-5f5027963671 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:14.823937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1148/2300] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:14 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:14.826732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45d7e4-daa7-433a-9677-6765426d17d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:14.827457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45d7e4-daa7-433a-9677-6765426d17d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:14.827778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45d7e4-daa7-433a-9677-6765426d17d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:14.828002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45d7e4-daa7-433a-9677-6765426d17d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:14.828568 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c96db7-923f-46cb-ad7e-08c0789480cf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:14.828705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45d7e4-daa7-433a-9677-6765426d17d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:14.830835 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45d7e4-daa7-433a-9677-6765426d17d0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:14.833712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:14.875319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1153/2301] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:14 2022] POST /identity/v3/projects => generated 365 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:14.876230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45d7e4-daa7-433a-9677-6765426d17d0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:14.910052 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:14.932980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c96db7-923f-46cb-ad7e-08c0789480cf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:14.996817 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45d7e4-daa7-433a-9677-6765426d17d0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.003529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ede6fd-f457-4200-95b3-c5a8facb9c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.004447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ede6fd-f457-4200-95b3-c5a8facb9c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.004962 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ede6fd-f457-4200-95b3-c5a8facb9c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.005433 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ede6fd-f457-4200-95b3-c5a8facb9c06 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.006913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ede6fd-f457-4200-95b3-c5a8facb9c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.009282 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ede6fd-f457-4200-95b3-c5a8facb9c06 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.049799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1154/2302] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:14 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:15.053348 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ede6fd-f457-4200-95b3-c5a8facb9c06 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.069623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e3e15-26ec-4dec-8c24-1c60e1b0aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.070999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e3e15-26ec-4dec-8c24-1c60e1b0aa1c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.071840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e3e15-26ec-4dec-8c24-1c60e1b0aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.072410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e3e15-26ec-4dec-8c24-1c60e1b0aa1c admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.073454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e3e15-26ec-4dec-8c24-1c60e1b0aa1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.076079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e3e15-26ec-4dec-8c24-1c60e1b0aa1c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.077456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ede6fd-f457-4200-95b3-c5a8facb9c06 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.087940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a790-a815-47fa-924a-efc55715a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.088701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a790-a815-47fa-924a-efc55715a2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.088990 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a790-a815-47fa-924a-efc55715a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.089260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a790-a815-47fa-924a-efc55715a2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.089905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16a790-a815-47fa-924a-efc55715a2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.091622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16a790-a815-47fa-924a-efc55715a2d3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.118259 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1155/2303] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:15.119501 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1149/2304] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:14 2022] POST /identity/v3/users => generated 285 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:15.120079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a790-a815-47fa-924a-efc55715a2d3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.121034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e3e15-26ec-4dec-8c24-1c60e1b0aa1c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.135232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e3e15-26ec-4dec-8c24-1c60e1b0aa1c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.141638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e7bc5-01e0-4ccd-96b0-faff9b975a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.142281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a790-a815-47fa-924a-efc55715a2d3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.143142 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e7bc5-01e0-4ccd-96b0-faff9b975a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.143719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e7bc5-01e0-4ccd-96b0-faff9b975a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.144200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e7bc5-01e0-4ccd-96b0-faff9b975a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.145981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e7bc5-01e0-4ccd-96b0-faff9b975a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.148205 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e7bc5-01e0-4ccd-96b0-faff9b975a90 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.153733 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9240-5cea-4a1b-a900-6e9d3b50c596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.154431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9240-5cea-4a1b-a900-6e9d3b50c596 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.154714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9240-5cea-4a1b-a900-6e9d3b50c596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.154986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9240-5cea-4a1b-a900-6e9d3b50c596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.155664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c9240-5cea-4a1b-a900-6e9d3b50c596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.157800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c9240-5cea-4a1b-a900-6e9d3b50c596 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.162379 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1150/2305] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:15.166240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e7bc5-01e0-4ccd-96b0-faff9b975a90 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.179771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1156/2306] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:15.181255 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9240-5cea-4a1b-a900-6e9d3b50c596 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.191150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e7bc5-01e0-4ccd-96b0-faff9b975a90 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.201568 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9240-5cea-4a1b-a900-6e9d3b50c596 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.204183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e899cb-a816-4632-aad1-2c5726f75ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.205326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e899cb-a816-4632-aad1-2c5726f75ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.205602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e899cb-a816-4632-aad1-2c5726f75ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.206103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e899cb-a816-4632-aad1-2c5726f75ef1 admin admin] PATH_INFO: `/v3/projects/83331ca1ffa34767a96ef4455699a661/users/15c9f3b2a2284673ad9e7ed965051d78/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.215165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74132d91-3a3a-47a4-a98f-e66b53598fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.215871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74132d91-3a3a-47a4-a98f-e66b53598fab admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.216228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74132d91-3a3a-47a4-a98f-e66b53598fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.216502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74132d91-3a3a-47a4-a98f-e66b53598fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.217786 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74132d91-3a3a-47a4-a98f-e66b53598fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.219831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74132d91-3a3a-47a4-a98f-e66b53598fab admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.227796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e899cb-a816-4632-aad1-2c5726f75ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83331ca1ffa34767a96ef4455699a661, user_id=15c9f3b2a2284673ad9e7ed965051d78, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.231309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e899cb-a816-4632-aad1-2c5726f75ef1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.240057 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1157/2307] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:15.241674 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74132d91-3a3a-47a4-a98f-e66b53598fab admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.249231 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1151/2308] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:15 2022] PUT /identity/v3/projects/83331ca1ffa34767a96ef4455699a661/users/15c9f3b2a2284673ad9e7ed965051d78/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:15.262515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522fa41-aaca-463f-8e04-d500d5273bd1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.263147 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522fa41-aaca-463f-8e04-d500d5273bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.263947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522fa41-aaca-463f-8e04-d500d5273bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.264256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74132d91-3a3a-47a4-a98f-e66b53598fab admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.328655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9522fa41-aaca-463f-8e04-d500d5273bd1 None None] MFA Rules not processed for user `15c9f3b2a2284673ad9e7ed965051d78`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:15.330038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9522fa41-aaca-463f-8e04-d500d5273bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:15.368896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f6d2c-8b70-44a8-9e82-79df358ff3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.369825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f6d2c-8b70-44a8-9e82-79df358ff3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.370204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f6d2c-8b70-44a8-9e82-79df358ff3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.370628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f6d2c-8b70-44a8-9e82-79df358ff3f8 admin admin] PATH_INFO: `/v3/projects/2ccf0ce3083345289d1d12141d0ad271/users/56c2490e8d6f4bf9a20ae0db29400181/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.389216 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f6d2c-8b70-44a8-9e82-79df358ff3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccf0ce3083345289d1d12141d0ad271, user_id=56c2490e8d6f4bf9a20ae0db29400181, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.392094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f6d2c-8b70-44a8-9e82-79df358ff3f8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.410386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1158/2309] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:15 2022] PUT /identity/v3/projects/2ccf0ce3083345289d1d12141d0ad271/users/56c2490e8d6f4bf9a20ae0db29400181/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:15.418666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deebab4-d557-4cb6-b8d1-7a892340dcb6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.419048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deebab4-d557-4cb6-b8d1-7a892340dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.419400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deebab4-d557-4cb6-b8d1-7a892340dcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.422671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:15.439408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1152/2310] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:15.445657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522fa41-aaca-463f-8e04-d500d5273bd1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.460467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522fa41-aaca-463f-8e04-d500d5273bd1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.462843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7deebab4-d557-4cb6-b8d1-7a892340dcb6 None None] MFA Rules not processed for user `56c2490e8d6f4bf9a20ae0db29400181`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:15.464102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7deebab4-d557-4cb6-b8d1-7a892340dcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:15.513623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:15.524517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcc161-18ba-46fa-b3d1-ade43f04831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.525230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcc161-18ba-46fa-b3d1-ade43f04831a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.525469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcc161-18ba-46fa-b3d1-ade43f04831a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.525751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcc161-18ba-46fa-b3d1-ade43f04831a admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.526433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcc161-18ba-46fa-b3d1-ade43f04831a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.528314 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcc161-18ba-46fa-b3d1-ade43f04831a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.533480 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1159/2311] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:15.538575 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deebab4-d557-4cb6-b8d1-7a892340dcb6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.549146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deebab4-d557-4cb6-b8d1-7a892340dcb6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.554235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8493e0-fb49-44e5-a479-16b22bb1e6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.554861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8493e0-fb49-44e5-a479-16b22bb1e6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.555133 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8493e0-fb49-44e5-a479-16b22bb1e6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.555399 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8493e0-fb49-44e5-a479-16b22bb1e6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.556023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8493e0-fb49-44e5-a479-16b22bb1e6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.557916 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8493e0-fb49-44e5-a479-16b22bb1e6e2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.558276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1153/2312] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:15 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:15.562636 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcc161-18ba-46fa-b3d1-ade43f04831a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.591662 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1160/2313] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:15 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:15.595253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8493e0-fb49-44e5-a479-16b22bb1e6e2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.627192 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcc161-18ba-46fa-b3d1-ade43f04831a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.657608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8493e0-fb49-44e5-a479-16b22bb1e6e2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.662895 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ffb8e-3739-47f0-8c5c-da7779b0e1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.663491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ffb8e-3739-47f0-8c5c-da7779b0e1bf admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.663767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ffb8e-3739-47f0-8c5c-da7779b0e1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.664036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ffb8e-3739-47f0-8c5c-da7779b0e1bf admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.664659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ffb8e-3739-47f0-8c5c-da7779b0e1bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.666652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ffb8e-3739-47f0-8c5c-da7779b0e1bf admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.684529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b7bc7-e4f9-446b-ab2a-24ad34390158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.685130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b7bc7-e4f9-446b-ab2a-24ad34390158 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.685406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b7bc7-e4f9-446b-ab2a-24ad34390158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.685672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b7bc7-e4f9-446b-ab2a-24ad34390158 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.686290 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b7bc7-e4f9-446b-ab2a-24ad34390158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.687539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1161/2314] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:15 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:15.688212 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b7bc7-e4f9-446b-ab2a-24ad34390158 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.691234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ffb8e-3739-47f0-8c5c-da7779b0e1bf admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.701780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ffb8e-3739-47f0-8c5c-da7779b0e1bf admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.707046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9eaa5-3e40-4ae6-aee4-aa3a5f253403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.708273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9eaa5-3e40-4ae6-aee4-aa3a5f253403 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.708551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9eaa5-3e40-4ae6-aee4-aa3a5f253403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.708825 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9eaa5-3e40-4ae6-aee4-aa3a5f253403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.709461 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9eaa5-3e40-4ae6-aee4-aa3a5f253403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.709850 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1154/2315] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:15 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:15.711207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9eaa5-3e40-4ae6-aee4-aa3a5f253403 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.714153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b7bc7-e4f9-446b-ab2a-24ad34390158 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.725859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1162/2316] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:15.726040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b7bc7-e4f9-446b-ab2a-24ad34390158 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.727209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9eaa5-3e40-4ae6-aee4-aa3a5f253403 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.730368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc1aeb-b75f-4d9f-8e95-b8b727d85917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.730946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc1aeb-b75f-4d9f-8e95-b8b727d85917 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.731219 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc1aeb-b75f-4d9f-8e95-b8b727d85917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.731487 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc1aeb-b75f-4d9f-8e95-b8b727d85917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.732127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc1aeb-b75f-4d9f-8e95-b8b727d85917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.733699 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc1aeb-b75f-4d9f-8e95-b8b727d85917 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.737493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9eaa5-3e40-4ae6-aee4-aa3a5f253403 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.742810 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f292-608b-47e6-8615-508a53201826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.743452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f292-608b-47e6-8615-508a53201826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.743759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f292-608b-47e6-8615-508a53201826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.744038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f292-608b-47e6-8615-508a53201826 admin admin] PATH_INFO: `/v3/projects/0004d6f0692742f081101a69778733ac/users/d3b854a839c348e7a5e0686bdef3d3eb/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.744157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1155/2317] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:15.747993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc1aeb-b75f-4d9f-8e95-b8b727d85917 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.754691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:15.758872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc1aeb-b75f-4d9f-8e95-b8b727d85917 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.760470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03f292-608b-47e6-8615-508a53201826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0004d6f0692742f081101a69778733ac, user_id=d3b854a839c348e7a5e0686bdef3d3eb, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.763524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03f292-608b-47e6-8615-508a53201826 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.764553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b0467-35bf-4986-9d4b-e7b447342ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.765133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b0467-35bf-4986-9d4b-e7b447342ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.765411 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b0467-35bf-4986-9d4b-e7b447342ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.765686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b0467-35bf-4986-9d4b-e7b447342ebd admin admin] PATH_INFO: `/v3/projects/3dac4b963a7340519761db7d57fbcfdf/users/432f0a2b90fa4df9825091c0bea63049/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.777449 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1163/2318] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:15 2022] PUT /identity/v3/projects/0004d6f0692742f081101a69778733ac/users/d3b854a839c348e7a5e0686bdef3d3eb/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:15.781191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b0467-35bf-4986-9d4b-e7b447342ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dac4b963a7340519761db7d57fbcfdf, user_id=432f0a2b90fa4df9825091c0bea63049, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.781352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f292-608b-47e6-8615-508a53201826 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.783993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b0467-35bf-4986-9d4b-e7b447342ebd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.798709 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1156/2319] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:15 2022] PUT /identity/v3/projects/3dac4b963a7340519761db7d57fbcfdf/users/432f0a2b90fa4df9825091c0bea63049/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:15.802576 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b0467-35bf-4986-9d4b-e7b447342ebd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.848317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f292-608b-47e6-8615-508a53201826 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.864877 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b0467-35bf-4986-9d4b-e7b447342ebd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.869787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048016-1047-4806-ad08-c057f340f224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.870420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048016-1047-4806-ad08-c057f340f224 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.870702 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048016-1047-4806-ad08-c057f340f224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.870976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048016-1047-4806-ad08-c057f340f224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.871628 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9048016-1047-4806-ad08-c057f340f224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.873322 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9048016-1047-4806-ad08-c057f340f224 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.884347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:15.887790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1157/2320] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:15.888549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048016-1047-4806-ad08-c057f340f224 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.899300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048016-1047-4806-ad08-c057f340f224 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.905022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641690f-b56d-49cb-bcc3-7c0c9202fd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.905672 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641690f-b56d-49cb-bcc3-7c0c9202fd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.905977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641690f-b56d-49cb-bcc3-7c0c9202fd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.906256 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641690f-b56d-49cb-bcc3-7c0c9202fd89 admin admin] PATH_INFO: `/v3/projects/3dac4b963a7340519761db7d57fbcfdf/users/432f0a2b90fa4df9825091c0bea63049/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.908229 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641690f-b56d-49cb-bcc3-7c0c9202fd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dac4b963a7340519761db7d57fbcfdf, user_id=432f0a2b90fa4df9825091c0bea63049, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.910979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641690f-b56d-49cb-bcc3-7c0c9202fd89 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.916955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fbf5-5d59-4150-9608-2ddbf52007fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:15.917861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fbf5-5d59-4150-9608-2ddbf52007fd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.918139 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fbf5-5d59-4150-9608-2ddbf52007fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.918439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fbf5-5d59-4150-9608-2ddbf52007fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.919238 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069fbf5-5d59-4150-9608-2ddbf52007fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:15.920898 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069fbf5-5d59-4150-9608-2ddbf52007fd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:15.922106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1158/2321] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:15 2022] PUT /identity/v3/projects/3dac4b963a7340519761db7d57fbcfdf/users/432f0a2b90fa4df9825091c0bea63049/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:15.932067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c7f04-abc4-4e6d-a685-1279d4b879a6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:15.932349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c7f04-abc4-4e6d-a685-1279d4b879a6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:15.932627 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c7f04-abc4-4e6d-a685-1279d4b879a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:15.936192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1164/2322] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:15.937189 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fbf5-5d59-4150-9608-2ddbf52007fd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:15.948260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fbf5-5d59-4150-9608-2ddbf52007fd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:15.966831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-134c7f04-abc4-4e6d-a685-1279d4b879a6 None None] MFA Rules not processed for user `432f0a2b90fa4df9825091c0bea63049`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:15.968157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-134c7f04-abc4-4e6d-a685-1279d4b879a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:16.014880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef362c-405b-47b7-9f47-b4d395326328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:16.015475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef362c-405b-47b7-9f47-b4d395326328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:16.015755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef362c-405b-47b7-9f47-b4d395326328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:16.016563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef362c-405b-47b7-9f47-b4d395326328 admin admin] PATH_INFO: `/v3/projects/0004d6f0692742f081101a69778733ac/users/d3b854a839c348e7a5e0686bdef3d3eb/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:16.018779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef362c-405b-47b7-9f47-b4d395326328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0004d6f0692742f081101a69778733ac, user_id=d3b854a839c348e7a5e0686bdef3d3eb, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:16.022015 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef362c-405b-47b7-9f47-b4d395326328 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:16.041271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1165/2323] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:15 2022] PUT /identity/v3/projects/0004d6f0692742f081101a69778733ac/users/d3b854a839c348e7a5e0686bdef3d3eb/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:16.051041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1159/2324] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:16.051549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224f6d3-146b-4b0f-bc4d-3515ac68ebae None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:16.051841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224f6d3-146b-4b0f-bc4d-3515ac68ebae None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:16.052113 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224f6d3-146b-4b0f-bc4d-3515ac68ebae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:16.063028 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c7f04-abc4-4e6d-a685-1279d4b879a6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:16.064748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-134c7f04-abc4-4e6d-a685-1279d4b879a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:16.084571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3224f6d3-146b-4b0f-bc4d-3515ac68ebae None None] MFA Rules not processed for user `d3b854a839c348e7a5e0686bdef3d3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:16.085778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3224f6d3-146b-4b0f-bc4d-3515ac68ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:16.120129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:16.123873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c7f04-abc4-4e6d-a685-1279d4b879a6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:16.156540 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1166/2325] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:16.174303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224f6d3-146b-4b0f-bc4d-3515ac68ebae None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:16.174800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147bfc7-90cf-43de-99ce-2fd79a73aa31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:16.175433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147bfc7-90cf-43de-99ce-2fd79a73aa31 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:16.175694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3224f6d3-146b-4b0f-bc4d-3515ac68ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:16.175800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147bfc7-90cf-43de-99ce-2fd79a73aa31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:16.176547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147bfc7-90cf-43de-99ce-2fd79a73aa31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:16.188103 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224f6d3-146b-4b0f-bc4d-3515ac68ebae None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:16.192242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b05c1-b5a1-4334-a3a7-8242c5859593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:16.192846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b05c1-b5a1-4334-a3a7-8242c5859593 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:16.193122 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b05c1-b5a1-4334-a3a7-8242c5859593 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:16.193402 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b05c1-b5a1-4334-a3a7-8242c5859593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:16.202580 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b05c1-b5a1-4334-a3a7-8242c5859593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:16.204016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b05c1-b5a1-4334-a3a7-8242c5859593 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:16.213364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1167/2326] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:16.239324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147bfc7-90cf-43de-99ce-2fd79a73aa31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:16.240781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147bfc7-90cf-43de-99ce-2fd79a73aa31 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:16.250986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1160/2327] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:16.260001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b05c1-b5a1-4334-a3a7-8242c5859593 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:16.268593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b05c1-b5a1-4334-a3a7-8242c5859593 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:16.272264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9db7-eb76-405c-aff7-3f2f75e0cdab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:16.272853 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9db7-eb76-405c-aff7-3f2f75e0cdab service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:16.273126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9db7-eb76-405c-aff7-3f2f75e0cdab service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:16.273776 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9db7-eb76-405c-aff7-3f2f75e0cdab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:16.305772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:16.335595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b9db7-eb76-405c-aff7-3f2f75e0cdab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:16.337111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b9db7-eb76-405c-aff7-3f2f75e0cdab service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:16.337688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147bfc7-90cf-43de-99ce-2fd79a73aa31 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:16.346262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147bfc7-90cf-43de-99ce-2fd79a73aa31 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:16.346454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1168/2328] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:16.350179 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597efb11-39b8-473c-b348-fa4e53bfea84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:16.350806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597efb11-39b8-473c-b348-fa4e53bfea84 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:16.351218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597efb11-39b8-473c-b348-fa4e53bfea84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:16.351320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597efb11-39b8-473c-b348-fa4e53bfea84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:16.413402 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597efb11-39b8-473c-b348-fa4e53bfea84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:16.414921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597efb11-39b8-473c-b348-fa4e53bfea84 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:16.423839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1161/2329] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:19.752300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9db7-eb76-405c-aff7-3f2f75e0cdab service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:19.816405 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9db7-eb76-405c-aff7-3f2f75e0cdab service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:19.821856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:19.822498 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:19.822773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:19.823041 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] PATH_INFO: `/v3/users/56c2490e8d6f4bf9a20ae0db29400181` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:19.824146 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c2490e8d6f4bf9a20ae0db29400181)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:19.826038 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:19.878262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c2490e8d6f4bf9a20ae0db29400181'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:19.878952 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c2490e8d6f4bf9a20ae0db29400181'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:19.891256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c2490e8d6f4bf9a20ae0db29400181'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:19.900781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c2490e8d6f4bf9a20ae0db29400181'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:19.901491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c2490e8d6f4bf9a20ae0db29400181'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:19.915049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1169/2330] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:19 2022] DELETE /identity/v3/users/56c2490e8d6f4bf9a20ae0db29400181 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:19.919795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab378f-504f-4835-8655-b76de66a3d59 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:19.920338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab378f-504f-4835-8655-b76de66a3d59 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:19.920524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab378f-504f-4835-8655-b76de66a3d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:19.946133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-56ab378f-504f-4835-8655-b76de66a3d59 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:19.947382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-56ab378f-504f-4835-8655-b76de66a3d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:19.999442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:20.014724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1162/2331] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:20.024481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:20.025908 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:20.088194 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e363a-5e81-4cdd-bec7-1a1c8c350ed2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:20.091780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7a0ec-52e9-42c7-90ca-7ebd0e370ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:20.092518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a0ec-52e9-42c7-90ca-7ebd0e370ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:20.092795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a0ec-52e9-42c7-90ca-7ebd0e370ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:20.093062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a0ec-52e9-42c7-90ca-7ebd0e370ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:20.098736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:20.102729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7a0ec-52e9-42c7-90ca-7ebd0e370ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:20.104440 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7a0ec-52e9-42c7-90ca-7ebd0e370ae1 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:20.116572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1170/2332] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:20.391983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab378f-504f-4835-8655-b76de66a3d59 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:20.393946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-56ab378f-504f-4835-8655-b76de66a3d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:20.406469 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab378f-504f-4835-8655-b76de66a3d59 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:20.411716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:20.412449 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:20.412824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:20.413185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] PATH_INFO: `/v3/projects/2ccf0ce3083345289d1d12141d0ad271` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:20.414435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccf0ce3083345289d1d12141d0ad271)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:20.416312 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:20.463382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccf0ce3083345289d1d12141d0ad271', 'request_id': 'req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.469460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccf0ce3083345289d1d12141d0ad271', 'request_id': 'req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.475765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccf0ce3083345289d1d12141d0ad271', 'request_id': 'req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.488381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] The token cache is being invalidate because project 2ccf0ce3083345289d1d12141d0ad271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:20.489278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.489888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.491388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1163/2333] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:20 2022] DELETE /identity/v3/projects/2ccf0ce3083345289d1d12141d0ad271 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:20.495317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7a0ec-52e9-42c7-90ca-7ebd0e370ae1 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:20.505481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7a0ec-52e9-42c7-90ca-7ebd0e370ae1 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:20.613178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:20.613526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:20.613526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:20.614473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] PATH_INFO: `/v3/users/d3b854a839c348e7a5e0686bdef3d3eb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:20.615649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b854a839c348e7a5e0686bdef3d3eb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:20.617630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:20.699797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b854a839c348e7a5e0686bdef3d3eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.700517 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b854a839c348e7a5e0686bdef3d3eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.713925 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b854a839c348e7a5e0686bdef3d3eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.721623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b854a839c348e7a5e0686bdef3d3eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.722301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-62c56c46-3552-4d09-ae5e-96de5da7462d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b854a839c348e7a5e0686bdef3d3eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:20.736723 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1171/2334] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:20 2022] DELETE /identity/v3/users/d3b854a839c348e7a5e0686bdef3d3eb => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:21.015546 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.017007 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.031096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf484ac-872b-4d0c-a947-c0b9265bc6c5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.095163 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.095795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.096203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.096319 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] PATH_INFO: `/v3/projects/0004d6f0692742f081101a69778733ac` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.097550 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0004d6f0692742f081101a69778733ac)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.099561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.150045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0004d6f0692742f081101a69778733ac', 'request_id': 'req-ee77eab0-f66d-4fb7-b141-8910ae12a941'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.156642 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0004d6f0692742f081101a69778733ac', 'request_id': 'req-ee77eab0-f66d-4fb7-b141-8910ae12a941'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.163444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0004d6f0692742f081101a69778733ac', 'request_id': 'req-ee77eab0-f66d-4fb7-b141-8910ae12a941'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.177407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] The token cache is being invalidate because project 0004d6f0692742f081101a69778733ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:21.177690 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.178333 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.179885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1164/2335] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:21 2022] DELETE /identity/v3/projects/0004d6f0692742f081101a69778733ac => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:21.184714 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbbed3-2cc2-4fea-95d6-44d6644cf04e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.185201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbbed3-2cc2-4fea-95d6-44d6644cf04e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.185314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbbed3-2cc2-4fea-95d6-44d6644cf04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.207696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-63fbbed3-2cc2-4fea-95d6-44d6644cf04e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:21.209002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-63fbbed3-2cc2-4fea-95d6-44d6644cf04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.229356 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.231045 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.242279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:21.278333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1172/2336] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:21.286389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f68c7d-ac1b-42a9-ad0f-016919b2bd3a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.286673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f68c7d-ac1b-42a9-ad0f-016919b2bd3a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.286948 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f68c7d-ac1b-42a9-ad0f-016919b2bd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.300415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77eab0-f66d-4fb7-b141-8910ae12a941 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.306048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.306708 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.306983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.307249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] PATH_INFO: `/v3/users/15c9f3b2a2284673ad9e7ed965051d78` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.308374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15c9f3b2a2284673ad9e7ed965051d78)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.309028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-80f68c7d-ac1b-42a9-ad0f-016919b2bd3a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:21.310229 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-80f68c7d-ac1b-42a9-ad0f-016919b2bd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.310391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.318633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1173/2337] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:21.322843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f68c7d-ac1b-42a9-ad0f-016919b2bd3a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.332966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f68c7d-ac1b-42a9-ad0f-016919b2bd3a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.338539 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.339130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.339515 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.339780 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.340516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:21.340824 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.342649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.344333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:21.357518 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1174/2338] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:35:21.360313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15c9f3b2a2284673ad9e7ed965051d78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.360991 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15c9f3b2a2284673ad9e7ed965051d78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.361805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.363871 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.369242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15c9f3b2a2284673ad9e7ed965051d78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.377021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521490-34d2-4a40-ba92-42c15bd58b72 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.381765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec7881-269b-4474-a247-6653061ad0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.381926 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15c9f3b2a2284673ad9e7ed965051d78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.382381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec7881-269b-4474-a247-6653061ad0dc admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.382489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b9f1f1f2-8932-45c9-a2f7-c71a124d7136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15c9f3b2a2284673ad9e7ed965051d78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:21.382665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec7881-269b-4474-a247-6653061ad0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.382937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec7881-269b-4474-a247-6653061ad0dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.383563 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec7881-269b-4474-a247-6653061ad0dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.385505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec7881-269b-4474-a247-6653061ad0dc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.395907 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1165/2339] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:21 2022] DELETE /identity/v3/users/15c9f3b2a2284673ad9e7ed965051d78 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:21.399831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f38c-0beb-40d0-9f4d-51c95825aac7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.400112 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f38c-0beb-40d0-9f4d-51c95825aac7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.400381 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f38c-0beb-40d0-9f4d-51c95825aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.414564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1175/2340] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:21 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:21.418647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec7881-269b-4474-a247-6653061ad0dc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.420355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4e44f38c-0beb-40d0-9f4d-51c95825aac7 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:21.421535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e44f38c-0beb-40d0-9f4d-51c95825aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.481504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1166/2341] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:21.483587 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec7881-269b-4474-a247-6653061ad0dc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.488223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d9136-22f1-4d40-ad65-ae8cb5359367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.489076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9136-22f1-4d40-ad65-ae8cb5359367 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.489355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9136-22f1-4d40-ad65-ae8cb5359367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.489630 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9136-22f1-4d40-ad65-ae8cb5359367 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.490263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d9136-22f1-4d40-ad65-ae8cb5359367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.491217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f38c-0beb-40d0-9f4d-51c95825aac7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.492219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d9136-22f1-4d40-ad65-ae8cb5359367 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.492585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e44f38c-0beb-40d0-9f4d-51c95825aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.518890 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1176/2342] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:21 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:21.520457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d9136-22f1-4d40-ad65-ae8cb5359367 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.525193 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:21.531120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d9136-22f1-4d40-ad65-ae8cb5359367 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.537242 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4b268-54ad-49c5-8900-576a9411448e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.537947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b268-54ad-49c5-8900-576a9411448e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.538265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b268-54ad-49c5-8900-576a9411448e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.538628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b268-54ad-49c5-8900-576a9411448e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.539473 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4b268-54ad-49c5-8900-576a9411448e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.541142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4b268-54ad-49c5-8900-576a9411448e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.551237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1177/2343] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:21.555688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4b268-54ad-49c5-8900-576a9411448e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.558565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f38c-0beb-40d0-9f4d-51c95825aac7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.562107 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f0c63-5924-4b5c-816b-6b982f1012f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.562731 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f0c63-5924-4b5c-816b-6b982f1012f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.563006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f0c63-5924-4b5c-816b-6b982f1012f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.563274 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f0c63-5924-4b5c-816b-6b982f1012f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.565633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4b268-54ad-49c5-8900-576a9411448e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.570431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79e2b-4822-4c26-b7ae-dd357279533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.571009 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79e2b-4822-4c26-b7ae-dd357279533e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.571287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79e2b-4822-4c26-b7ae-dd357279533e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.571558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79e2b-4822-4c26-b7ae-dd357279533e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.572187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79e2b-4822-4c26-b7ae-dd357279533e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.572492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f0c63-5924-4b5c-816b-6b982f1012f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.573762 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79e2b-4822-4c26-b7ae-dd357279533e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.573944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f0c63-5924-4b5c-816b-6b982f1012f6 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.584658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1167/2344] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:21.587913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1178/2345] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:21.589431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f0c63-5924-4b5c-816b-6b982f1012f6 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.599089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f0c63-5924-4b5c-816b-6b982f1012f6 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.603947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cd9b3-1015-4df0-9975-85d1bc634413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.604577 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cd9b3-1015-4df0-9975-85d1bc634413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.604849 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cd9b3-1015-4df0-9975-85d1bc634413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.605368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cd9b3-1015-4df0-9975-85d1bc634413 admin admin] PATH_INFO: `/v3/projects/eedae4768d604650aa58755b7db87641/users/acff4954e5f84db98783f4d831ca217d/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.620464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cd9b3-1015-4df0-9975-85d1bc634413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eedae4768d604650aa58755b7db87641, user_id=acff4954e5f84db98783f4d831ca217d, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.623320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cd9b3-1015-4df0-9975-85d1bc634413 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.637795 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1168/2346] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:21 2022] PUT /identity/v3/projects/eedae4768d604650aa58755b7db87641/users/acff4954e5f84db98783f4d831ca217d/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:21.649831 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f0bb0-a685-4ebd-919a-dc87a7351c9b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.650280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f0bb0-a685-4ebd-919a-dc87a7351c9b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.650665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f0bb0-a685-4ebd-919a-dc87a7351c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.680217 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8e3f0bb0-a685-4ebd-919a-dc87a7351c9b None None] MFA Rules not processed for user `acff4954e5f84db98783f4d831ca217d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:21.681418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e3f0bb0-a685-4ebd-919a-dc87a7351c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.751412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1179/2347] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:21.756914 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cd9b3-1015-4df0-9975-85d1bc634413 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.821101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cd9b3-1015-4df0-9975-85d1bc634413 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.827142 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.827750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.828026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.828298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.829017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.830999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.865250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1169/2348] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:21 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:21.869459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f0bb0-a685-4ebd-919a-dc87a7351c9b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.880953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f0bb0-a685-4ebd-919a-dc87a7351c9b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.883562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.885508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:21.909708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:21.954053 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bae27-3327-453a-b39e-bbf6fff8666e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.955119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bae27-3327-453a-b39e-bbf6fff8666e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.955119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bae27-3327-453a-b39e-bbf6fff8666e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.955441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bae27-3327-453a-b39e-bbf6fff8666e admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.956334 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bae27-3327-453a-b39e-bbf6fff8666e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.959112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bae27-3327-453a-b39e-bbf6fff8666e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.974635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa4539-b6b9-4184-8bf3-a9d633024ae8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:21.980977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:21.981686 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:21.982031 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:21.982365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] PATH_INFO: `/v3/projects/83331ca1ffa34767a96ef4455699a661` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:21.983626 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=83331ca1ffa34767a96ef4455699a661)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:21.985649 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1180/2349] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:21 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:21.985803 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:21.990046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bae27-3327-453a-b39e-bbf6fff8666e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:21.999791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bae27-3327-453a-b39e-bbf6fff8666e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.010470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94203600-7a13-4bb1-a4ac-b460cf0f25ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.010470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94203600-7a13-4bb1-a4ac-b460cf0f25ff admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.010470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94203600-7a13-4bb1-a4ac-b460cf0f25ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.010470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94203600-7a13-4bb1-a4ac-b460cf0f25ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.010848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94203600-7a13-4bb1-a4ac-b460cf0f25ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.010848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94203600-7a13-4bb1-a4ac-b460cf0f25ff admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.025043 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:22.031698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1181/2350] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:22.036601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94203600-7a13-4bb1-a4ac-b460cf0f25ff admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.042388 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83331ca1ffa34767a96ef4455699a661', 'request_id': 'req-0dae15bc-246a-4618-95b2-a1daf7e43ddd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.046469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94203600-7a13-4bb1-a4ac-b460cf0f25ff admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.048724 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83331ca1ffa34767a96ef4455699a661', 'request_id': 'req-0dae15bc-246a-4618-95b2-a1daf7e43ddd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.055016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83331ca1ffa34767a96ef4455699a661', 'request_id': 'req-0dae15bc-246a-4618-95b2-a1daf7e43ddd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.068392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] The token cache is being invalidate because project 83331ca1ffa34767a96ef4455699a661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:22.068691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.069344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.071264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1170/2351] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:21 2022] DELETE /identity/v3/projects/83331ca1ffa34767a96ef4455699a661 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:22.074894 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.076247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.111037 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.111700 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.112008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.112281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] PATH_INFO: `/v3/projects/224a351e267f4e5680b366b817caa562/users/f190abaa24a948649b3fb701ac05e99d/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.126869 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] RBAC: Authorizing `identity:create_grant(project_id=224a351e267f4e5680b366b817caa562, user_id=f190abaa24a948649b3fb701ac05e99d, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.130206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.142305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae15bc-246a-4618-95b2-a1daf7e43ddd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.143345 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1182/2352] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:22 2022] PUT /identity/v3/projects/224a351e267f4e5680b366b817caa562/users/f190abaa24a948649b3fb701ac05e99d/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:22.147276 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.148899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.200640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.201182 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.201464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.201740 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] PATH_INFO: `/v3/users/432f0a2b90fa4df9825091c0bea63049` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.203098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=432f0a2b90fa4df9825091c0bea63049)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.205016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.212995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7db3e-29f1-41e8-a963-77e83363725b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.217854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585082c-aed6-4571-8507-0a5535234795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.218257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585082c-aed6-4571-8507-0a5535234795 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.218573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585082c-aed6-4571-8507-0a5535234795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.218855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585082c-aed6-4571-8507-0a5535234795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.219496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585082c-aed6-4571-8507-0a5535234795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.221085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585082c-aed6-4571-8507-0a5535234795 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.231610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1183/2353] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:22.235906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585082c-aed6-4571-8507-0a5535234795 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.247013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585082c-aed6-4571-8507-0a5535234795 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.251930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3724c4-183d-43dc-8a7e-36c0d4c8ea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.252565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3724c4-183d-43dc-8a7e-36c0d4c8ea47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.252845 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3724c4-183d-43dc-8a7e-36c0d4c8ea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.253120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3724c4-183d-43dc-8a7e-36c0d4c8ea47 admin admin] PATH_INFO: `/v3/projects/224a351e267f4e5680b366b817caa562/users/f190abaa24a948649b3fb701ac05e99d/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.255731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3724c4-183d-43dc-8a7e-36c0d4c8ea47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224a351e267f4e5680b366b817caa562, user_id=f190abaa24a948649b3fb701ac05e99d, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.256857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '432f0a2b90fa4df9825091c0bea63049'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.257597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '432f0a2b90fa4df9825091c0bea63049'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.258459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3724c4-183d-43dc-8a7e-36c0d4c8ea47 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.266403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '432f0a2b90fa4df9825091c0bea63049'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.270750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1184/2354] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:22 2022] PUT /identity/v3/projects/224a351e267f4e5680b366b817caa562/users/f190abaa24a948649b3fb701ac05e99d/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:22.278661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:22.280541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '432f0a2b90fa4df9825091c0bea63049'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.281397 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb4ee3-a961-43e4-9515-4cb59b31f4e4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.281544 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '432f0a2b90fa4df9825091c0bea63049'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.281700 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb4ee3-a961-43e4-9515-4cb59b31f4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.282049 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb4ee3-a961-43e4-9515-4cb59b31f4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.297247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1171/2355] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:22 2022] DELETE /identity/v3/users/432f0a2b90fa4df9825091c0bea63049 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:22.310006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:22.313208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e2fb4ee3-a961-43e4-9515-4cb59b31f4e4 None None] MFA Rules not processed for user `f190abaa24a948649b3fb701ac05e99d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:22.315046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2fb4ee3-a961-43e4-9515-4cb59b31f4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.389788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1185/2356] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:22.402837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.404299 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.467318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cfdfe-a1f2-4e9a-9225-f03f7b916730 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.471471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.472126 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.472401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.473221 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.483277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.484690 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.495545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1172/2357] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:22.562865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb4ee3-a961-43e4-9515-4cb59b31f4e4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.564656 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2fb4ee3-a961-43e4-9515-4cb59b31f4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.590482 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.592065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.650004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb4ee3-a961-43e4-9515-4cb59b31f4e4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.654805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.655543 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.655926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.656327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] PATH_INFO: `/v3/users/acff4954e5f84db98783f4d831ca217d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.657412 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acff4954e5f84db98783f4d831ca217d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.659375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.672896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f4898-17c1-4d2d-9f20-8c3da6df0a41 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.678118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.678858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.679174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.679509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] PATH_INFO: `/v3/projects/3dac4b963a7340519761db7d57fbcfdf` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.680701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dac4b963a7340519761db7d57fbcfdf)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.682669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.708048 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acff4954e5f84db98783f4d831ca217d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.708731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acff4954e5f84db98783f4d831ca217d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.721445 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acff4954e5f84db98783f4d831ca217d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.729456 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acff4954e5f84db98783f4d831ca217d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.730567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcd3d16-6f19-4314-ac14-dc06e16444c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acff4954e5f84db98783f4d831ca217d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.734084 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dac4b963a7340519761db7d57fbcfdf', 'request_id': 'req-10bc9338-2df8-4241-a1e5-eb7c510caa71'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.740647 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dac4b963a7340519761db7d57fbcfdf', 'request_id': 'req-10bc9338-2df8-4241-a1e5-eb7c510caa71'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.743775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1186/2358] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:22 2022] DELETE /identity/v3/users/acff4954e5f84db98783f4d831ca217d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:22.746827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dac4b963a7340519761db7d57fbcfdf', 'request_id': 'req-10bc9338-2df8-4241-a1e5-eb7c510caa71'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.748093 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dad47-f93c-4f81-acf8-096b3614beb0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.748421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dad47-f93c-4f81-acf8-096b3614beb0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.748771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dad47-f93c-4f81-acf8-096b3614beb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.761852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] The token cache is being invalidate because project 3dac4b963a7340519761db7d57fbcfdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:22.762133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.762789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-10bc9338-2df8-4241-a1e5-eb7c510caa71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:22.764279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1173/2359] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:22 2022] DELETE /identity/v3/projects/3dac4b963a7340519761db7d57fbcfdf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:22.770846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fbe85-065b-434b-922a-6fd8a9913b0f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.770846 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fbe85-065b-434b-922a-6fd8a9913b0f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.771102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fbe85-065b-434b-922a-6fd8a9913b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.772571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8c5dad47-f93c-4f81-acf8-096b3614beb0 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:22.773760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8c5dad47-f93c-4f81-acf8-096b3614beb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.785857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:22.791437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3d5fbe85-065b-434b-922a-6fd8a9913b0f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:22.792705 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3d5fbe85-065b-434b-922a-6fd8a9913b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.806667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:22.837709 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1187/2360] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:22.847533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dad47-f93c-4f81-acf8-096b3614beb0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.849209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8c5dad47-f93c-4f81-acf8-096b3614beb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.857013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1174/2361] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:22.864950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa7cfa-5d91-4503-88e7-cde0f9c56a47 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.865233 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa7cfa-5d91-4503-88e7-cde0f9c56a47 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.865509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa7cfa-5d91-4503-88e7-cde0f9c56a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.885892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-20fa7cfa-5d91-4503-88e7-cde0f9c56a47 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:22.887114 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-20fa7cfa-5d91-4503-88e7-cde0f9c56a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:22.895277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1175/2362] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:22.899650 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa7cfa-5d91-4503-88e7-cde0f9c56a47 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.908691 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dad47-f93c-4f81-acf8-096b3614beb0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.909104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa7cfa-5d91-4503-88e7-cde0f9c56a47 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.912343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee939b0-dfff-48d4-b692-1295cac81e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.912941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee939b0-dfff-48d4-b692-1295cac81e89 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.913212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee939b0-dfff-48d4-b692-1295cac81e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.913504 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee939b0-dfff-48d4-b692-1295cac81e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.914153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.914835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.915181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.915518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.916230 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:22.916718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.918387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.922452 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee939b0-dfff-48d4-b692-1295cac81e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.923866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee939b0-dfff-48d4-b692-1295cac81e89 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.930805 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1176/2363] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:35:22.934591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1188/2364] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:22.935016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.945581 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ece93-6a04-4309-86a2-bde01a35efc0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:22.950338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0ddce-b554-4284-89b8-5eb0716a9a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:22.951015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ddce-b554-4284-89b8-5eb0716a9a85 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:22.951218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ddce-b554-4284-89b8-5eb0716a9a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:22.951481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ddce-b554-4284-89b8-5eb0716a9a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:22.952103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0ddce-b554-4284-89b8-5eb0716a9a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:22.953967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0ddce-b554-4284-89b8-5eb0716a9a85 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:22.985705 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1177/2365] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:22 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:22.987025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee939b0-dfff-48d4-b692-1295cac81e89 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:22.997975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee939b0-dfff-48d4-b692-1295cac81e89 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:23.062142 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caba53-ba21-46fe-af04-4a2bc00d5454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:23.062970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caba53-ba21-46fe-af04-4a2bc00d5454 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.063288 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caba53-ba21-46fe-af04-4a2bc00d5454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.063652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caba53-ba21-46fe-af04-4a2bc00d5454 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.064395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caba53-ba21-46fe-af04-4a2bc00d5454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:23.066537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caba53-ba21-46fe-af04-4a2bc00d5454 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:23.094389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1189/2366] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:22 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:23.096508 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0ddce-b554-4284-89b8-5eb0716a9a85 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:23.110280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0ddce-b554-4284-89b8-5eb0716a9a85 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:23.116414 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275629f-bede-4325-900f-b47d4306a471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:23.117359 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275629f-bede-4325-900f-b47d4306a471 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.117783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275629f-bede-4325-900f-b47d4306a471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.118042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275629f-bede-4325-900f-b47d4306a471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.118840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275629f-bede-4325-900f-b47d4306a471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:23.120529 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275629f-bede-4325-900f-b47d4306a471 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:23.135115 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1178/2367] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:23.136735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caba53-ba21-46fe-af04-4a2bc00d5454 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:23.148318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caba53-ba21-46fe-af04-4a2bc00d5454 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:23.153379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:23.153992 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.154269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.154569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.155207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:23.156812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:23.170909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1190/2368] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:23.172497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275629f-bede-4325-900f-b47d4306a471 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:23.183655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275629f-bede-4325-900f-b47d4306a471 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:23.189418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee461bce-7dc8-4be2-a619-982a2aaacbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:23.190080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461bce-7dc8-4be2-a619-982a2aaacbf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.190451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461bce-7dc8-4be2-a619-982a2aaacbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.192061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461bce-7dc8-4be2-a619-982a2aaacbf4 admin admin] PATH_INFO: `/v3/projects/d7ec7ea91361407b9d2550cb13ade802/users/29f490bb81f640a4b2ff7e1d16313151/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.208458 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee461bce-7dc8-4be2-a619-982a2aaacbf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ec7ea91361407b9d2550cb13ade802, user_id=29f490bb81f640a4b2ff7e1d16313151, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:23.211682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee461bce-7dc8-4be2-a619-982a2aaacbf4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:23.227373 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:23.229857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:23.235050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1179/2369] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:23 2022] PUT /identity/v3/projects/d7ec7ea91361407b9d2550cb13ade802/users/29f490bb81f640a4b2ff7e1d16313151/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:23.246828 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0698e-8b13-4913-8fe0-f4b1c77200dc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.247175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0698e-8b13-4913-8fe0-f4b1c77200dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.247663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0698e-8b13-4913-8fe0-f4b1c77200dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.251880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2783626-58c7-41dd-a1ec-6b53852f6439 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:23.280050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-66c0698e-8b13-4913-8fe0-f4b1c77200dc None None] MFA Rules not processed for user `29f490bb81f640a4b2ff7e1d16313151`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:23.281270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-66c0698e-8b13-4913-8fe0-f4b1c77200dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:23.292421 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:23.313299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:23.313924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.314198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.314502 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] PATH_INFO: `/v3/projects/eedae4768d604650aa58755b7db87641` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.315643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eedae4768d604650aa58755b7db87641)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:23.317829 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:23.349070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1180/2370] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:23.360434 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0698e-8b13-4913-8fe0-f4b1c77200dc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:23.361796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-66c0698e-8b13-4913-8fe0-f4b1c77200dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:23.367386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eedae4768d604650aa58755b7db87641', 'request_id': 'req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.373621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eedae4768d604650aa58755b7db87641', 'request_id': 'req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.382215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eedae4768d604650aa58755b7db87641', 'request_id': 'req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.394028 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] The token cache is being invalidate because project eedae4768d604650aa58755b7db87641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:23.394317 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.395023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.396534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1191/2371] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:23 2022] DELETE /identity/v3/projects/eedae4768d604650aa58755b7db87641 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:23.399141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:23.400500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:23.401979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:23.424113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0698e-8b13-4913-8fe0-f4b1c77200dc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:23.467862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d7d42-8b17-49d5-b9b8-31145eba8c66 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:23.474096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:23.474520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ac934ed0122470cb489d6b324c664f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:23.474753 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.475027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.475133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.475294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] PATH_INFO: `/v3/users/f190abaa24a948649b3fb701ac05e99d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.475399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.475640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.476425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f190abaa24a948649b3fb701ac05e99d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:23.477481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:23.478503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:23.528620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f190abaa24a948649b3fb701ac05e99d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.529290 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f190abaa24a948649b3fb701ac05e99d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.539250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:23.540720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:23.542422 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f190abaa24a948649b3fb701ac05e99d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.550021 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1181/2372] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 10:35:23.550303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f190abaa24a948649b3fb701ac05e99d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.551743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c83cc5ba-6f88-422c-9a4b-0ea2cbacc833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f190abaa24a948649b3fb701ac05e99d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.553134 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:23.567877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1192/2373] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:23 2022] DELETE /identity/v3/users/f190abaa24a948649b3fb701ac05e99d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:23.850635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:23.864431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad65740-64f8-4adf-b062-e4314a386fe0 service glance] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:23.929180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:23.929808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:23.930085 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:23.930389 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] PATH_INFO: `/v3/projects/224a351e267f4e5680b366b817caa562` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:23.931519 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=224a351e267f4e5680b366b817caa562)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:23.933503 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:23.986661 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:23.990516 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224a351e267f4e5680b366b817caa562', 'request_id': 'req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:23.999067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224a351e267f4e5680b366b817caa562', 'request_id': 'req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:24.009980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224a351e267f4e5680b366b817caa562', 'request_id': 'req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:24.024750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] The token cache is being invalidate because project 224a351e267f4e5680b366b817caa562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:24.025058 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:24.025783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f63ccc9-9070-4a2e-9c23-69102a8ef2b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:24.027190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1182/2374] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:23 2022] DELETE /identity/v3/projects/224a351e267f4e5680b366b817caa562 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:24.032114 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770224a-8c22-48e1-bfd8-b34f36fbc8f9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.032406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770224a-8c22-48e1-bfd8-b34f36fbc8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.032741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770224a-8c22-48e1-bfd8-b34f36fbc8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.053079 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f770224a-8c22-48e1-bfd8-b34f36fbc8f9 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:24.054478 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f770224a-8c22-48e1-bfd8-b34f36fbc8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:24.115910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1193/2375] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:24.123990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7b5c-eebc-4347-b78d-a96db53b7d14 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.124471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7b5c-eebc-4347-b78d-a96db53b7d14 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.124572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7b5c-eebc-4347-b78d-a96db53b7d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.145391 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9c3c7b5c-eebc-4347-b78d-a96db53b7d14 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:24.146624 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9c3c7b5c-eebc-4347-b78d-a96db53b7d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:24.154756 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1183/2376] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:24.158987 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770224a-8c22-48e1-bfd8-b34f36fbc8f9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.168739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770224a-8c22-48e1-bfd8-b34f36fbc8f9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.173250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.173867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.174149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.174442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.175078 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:24.175378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.176995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.188819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1194/2377] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:35:24.193573 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c7b5c-eebc-4347-b78d-a96db53b7d14 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.202947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c7b5c-eebc-4347-b78d-a96db53b7d14 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.208165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb0905-1632-4516-a02c-3d1811ae033f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.208762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb0905-1632-4516-a02c-3d1811ae033f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.209030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb0905-1632-4516-a02c-3d1811ae033f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.209329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb0905-1632-4516-a02c-3d1811ae033f admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.209956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb0905-1632-4516-a02c-3d1811ae033f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.211960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb0905-1632-4516-a02c-3d1811ae033f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.239515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1184/2378] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:24 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:24.243704 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.253551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff6500-d43a-4df7-91e2-fe241be0956c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.314080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00133e7a-afb0-4564-9ffb-96b138d47da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.314730 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00133e7a-afb0-4564-9ffb-96b138d47da3 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.315011 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00133e7a-afb0-4564-9ffb-96b138d47da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.315284 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00133e7a-afb0-4564-9ffb-96b138d47da3 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.315956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00133e7a-afb0-4564-9ffb-96b138d47da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.318424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00133e7a-afb0-4564-9ffb-96b138d47da3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.340660 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1195/2379] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:24 2022] POST /identity/v3/users => generated 305 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:24.344960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb0905-1632-4516-a02c-3d1811ae033f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.361162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb0905-1632-4516-a02c-3d1811ae033f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.371033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4cca1-ee1c-4330-b072-cae0d6414011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.371242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4cca1-ee1c-4330-b072-cae0d6414011 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.371468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4cca1-ee1c-4330-b072-cae0d6414011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.371735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4cca1-ee1c-4330-b072-cae0d6414011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.372407 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4cca1-ee1c-4330-b072-cae0d6414011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.374006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4cca1-ee1c-4330-b072-cae0d6414011 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.387839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1185/2380] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:24.389061 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00133e7a-afb0-4564-9ffb-96b138d47da3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.399558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00133e7a-afb0-4564-9ffb-96b138d47da3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.405247 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decd3b9-d8f3-44c7-b470-a53f41b2efce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.405850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decd3b9-d8f3-44c7-b470-a53f41b2efce admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.406118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decd3b9-d8f3-44c7-b470-a53f41b2efce admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.406424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decd3b9-d8f3-44c7-b470-a53f41b2efce admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.407209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6decd3b9-d8f3-44c7-b470-a53f41b2efce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.409144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6decd3b9-d8f3-44c7-b470-a53f41b2efce admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.422380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1196/2381] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:24.426600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4cca1-ee1c-4330-b072-cae0d6414011 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.438969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4cca1-ee1c-4330-b072-cae0d6414011 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.444760 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93c41e-5120-4cc4-9ec4-e8cbf9dadf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.445549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c41e-5120-4cc4-9ec4-e8cbf9dadf30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.445868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c41e-5120-4cc4-9ec4-e8cbf9dadf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.446169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c41e-5120-4cc4-9ec4-e8cbf9dadf30 admin admin] PATH_INFO: `/v3/projects/ee1da2c8aed14147a7e76d0e6b5d1108/users/5a012a8b2caa48d891ec39abde6ff286/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.464228 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93c41e-5120-4cc4-9ec4-e8cbf9dadf30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1da2c8aed14147a7e76d0e6b5d1108, user_id=5a012a8b2caa48d891ec39abde6ff286, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.466933 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93c41e-5120-4cc4-9ec4-e8cbf9dadf30 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.482789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1186/2382] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:24 2022] PUT /identity/v3/projects/ee1da2c8aed14147a7e76d0e6b5d1108/users/5a012a8b2caa48d891ec39abde6ff286/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:24.493943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4a990-4515-43fd-865e-1f995fb89ef6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.494220 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4a990-4515-43fd-865e-1f995fb89ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.494523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4a990-4515-43fd-865e-1f995fb89ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.523434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f5b4a990-4515-43fd-865e-1f995fb89ef6 None None] MFA Rules not processed for user `5a012a8b2caa48d891ec39abde6ff286`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:24.524622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f5b4a990-4515-43fd-865e-1f995fb89ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:24.570429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:24.594879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1197/2383] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:24.600059 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93c41e-5120-4cc4-9ec4-e8cbf9dadf30 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.665916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93c41e-5120-4cc4-9ec4-e8cbf9dadf30 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.672110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbece51d-702c-4bab-803c-0084f3950614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.672714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece51d-702c-4bab-803c-0084f3950614 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.672988 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece51d-702c-4bab-803c-0084f3950614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.673260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece51d-702c-4bab-803c-0084f3950614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.673903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbece51d-702c-4bab-803c-0084f3950614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.675870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbece51d-702c-4bab-803c-0084f3950614 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.707845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1187/2384] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:24 2022] POST /identity/v3/projects => generated 409 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:35:24.711874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4a990-4515-43fd-865e-1f995fb89ef6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.730265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4a990-4515-43fd-865e-1f995fb89ef6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.793021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13214d-f132-4349-81ec-6681b036c173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.793637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13214d-f132-4349-81ec-6681b036c173 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.793911 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13214d-f132-4349-81ec-6681b036c173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.794193 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13214d-f132-4349-81ec-6681b036c173 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.794837 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13214d-f132-4349-81ec-6681b036c173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.796756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13214d-f132-4349-81ec-6681b036c173 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.819431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1198/2385] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:24 2022] POST /identity/v3/users => generated 311 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:24.823518 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbece51d-702c-4bab-803c-0084f3950614 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.834311 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbece51d-702c-4bab-803c-0084f3950614 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.839252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aeb0a-e38f-4875-9025-8b07732b2b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.839856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aeb0a-e38f-4875-9025-8b07732b2b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.840136 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aeb0a-e38f-4875-9025-8b07732b2b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.840409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aeb0a-e38f-4875-9025-8b07732b2b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.841052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6aeb0a-e38f-4875-9025-8b07732b2b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.842663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6aeb0a-e38f-4875-9025-8b07732b2b41 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.853965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1188/2386] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:24.858966 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13214d-f132-4349-81ec-6681b036c173 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.870941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13214d-f132-4349-81ec-6681b036c173 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.875604 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5cbec-ee6e-4f84-a618-abc2c66a87fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.876278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5cbec-ee6e-4f84-a618-abc2c66a87fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.876554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5cbec-ee6e-4f84-a618-abc2c66a87fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.876823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5cbec-ee6e-4f84-a618-abc2c66a87fe admin admin] PATH_INFO: `/v3/projects/c1af72a1e44b40328ccc0a819bc5f283/users/df562dbcd7ca42a18d03732d6ff81629/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.892096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5cbec-ee6e-4f84-a618-abc2c66a87fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1af72a1e44b40328ccc0a819bc5f283, user_id=df562dbcd7ca42a18d03732d6ff81629, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.894900 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5cbec-ee6e-4f84-a618-abc2c66a87fe admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.907647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1199/2387] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:24 2022] PUT /identity/v3/projects/c1af72a1e44b40328ccc0a819bc5f283/users/df562dbcd7ca42a18d03732d6ff81629/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:24.912976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aeb0a-e38f-4875-9025-8b07732b2b41 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:24.923532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aeb0a-e38f-4875-9025-8b07732b2b41 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:24.967075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:24.985574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e3c41-1010-4d52-9a59-56c0b02c4eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:24.986167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e3c41-1010-4d52-9a59-56c0b02c4eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:24.986471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e3c41-1010-4d52-9a59-56c0b02c4eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:24.986739 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e3c41-1010-4d52-9a59-56c0b02c4eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:24.987371 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e3c41-1010-4d52-9a59-56c0b02c4eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:24.989549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e3c41-1010-4d52-9a59-56c0b02c4eb3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:24.999127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1189/2388] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:25.003407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5cbec-ee6e-4f84-a618-abc2c66a87fe admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:25.014416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5cbec-ee6e-4f84-a618-abc2c66a87fe admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:25.019585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:25.021195 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:25.021534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:25.021809 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] PATH_INFO: `/v3/projects/c1af72a1e44b40328ccc0a819bc5f283/users/df562dbcd7ca42a18d03732d6ff81629/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:25.025176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1af72a1e44b40328ccc0a819bc5f283, user_id=df562dbcd7ca42a18d03732d6ff81629, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:25.026896 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:25.038083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1200/2389] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:25 2022] PUT /identity/v3/projects/c1af72a1e44b40328ccc0a819bc5f283/users/df562dbcd7ca42a18d03732d6ff81629/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:25.049728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f3251-9ef2-4d08-89df-06c6c7ba88c0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:25.050008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f3251-9ef2-4d08-89df-06c6c7ba88c0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:25.050282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f3251-9ef2-4d08-89df-06c6c7ba88c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:25.079635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cf3f3251-9ef2-4d08-89df-06c6c7ba88c0 None None] MFA Rules not processed for user `df562dbcd7ca42a18d03732d6ff81629`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:25.080769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cf3f3251-9ef2-4d08-89df-06c6c7ba88c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:25.153553 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1190/2390] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:25 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:25.164514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:25.166299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:25.225696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3b4f-f09c-46e9-8b4e-8d8d76ea65eb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:25.229351 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7641c-11dd-4457-98c2-ec4f91ad852f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:25.229951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7641c-11dd-4457-98c2-ec4f91ad852f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:25.230524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7641c-11dd-4457-98c2-ec4f91ad852f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:25.230797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7641c-11dd-4457-98c2-ec4f91ad852f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:25.240474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7641c-11dd-4457-98c2-ec4f91ad852f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:25.242352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7641c-11dd-4457-98c2-ec4f91ad852f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:25.252435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1201/2391] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:25 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:25.420771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f3251-9ef2-4d08-89df-06c6c7ba88c0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:25.432071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f3251-9ef2-4d08-89df-06c6c7ba88c0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:25.435654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580a465-1498-45cf-9b0d-2a457782ebc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:25.436248 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580a465-1498-45cf-9b0d-2a457782ebc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:25.436514 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580a465-1498-45cf-9b0d-2a457782ebc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:25.436806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580a465-1498-45cf-9b0d-2a457782ebc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:25.500736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580a465-1498-45cf-9b0d-2a457782ebc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:25.500736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580a465-1498-45cf-9b0d-2a457782ebc1 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:25.510152 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1191/2392] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:25 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:35.247843 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:35.505660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=71 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:38.017697 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7641c-11dd-4457-98c2-ec4f91ad852f service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:38.081495 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7641c-11dd-4457-98c2-ec4f91ad852f service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:38.085979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:38.086629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:38.086937 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:38.087314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] PATH_INFO: `/v3/users/5a012a8b2caa48d891ec39abde6ff286` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:38.088491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a012a8b2caa48d891ec39abde6ff286)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:38.090615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:38.141669 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a012a8b2caa48d891ec39abde6ff286'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.142308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a012a8b2caa48d891ec39abde6ff286'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.158909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a012a8b2caa48d891ec39abde6ff286'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.166858 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a012a8b2caa48d891ec39abde6ff286'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.167491 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a012a8b2caa48d891ec39abde6ff286'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.181301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1202/2393] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:38 2022] DELETE /identity/v3/users/5a012a8b2caa48d891ec39abde6ff286 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:38.185833 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd46c0-cd71-4dd4-bc67-595129b709a1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:38.186310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd46c0-cd71-4dd4-bc67-595129b709a1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:38.186471 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd46c0-cd71-4dd4-bc67-595129b709a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:38.205103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-60cd46c0-cd71-4dd4-bc67-595129b709a1 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:38.206497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60cd46c0-cd71-4dd4-bc67-595129b709a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:38.265937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1192/2394] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:38.276218 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:38.277633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:38.333972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dc672-f7db-419c-84f0-336f864ef93c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:38.337610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0567e-3fbb-4b54-82c4-d7524b4785fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:38.338205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0567e-3fbb-4b54-82c4-d7524b4785fe service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:38.338513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0567e-3fbb-4b54-82c4-d7524b4785fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:38.338808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0567e-3fbb-4b54-82c4-d7524b4785fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:38.347685 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0567e-3fbb-4b54-82c4-d7524b4785fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:38.349106 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0567e-3fbb-4b54-82c4-d7524b4785fe service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:38.358918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1203/2395] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:38.635369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd46c0-cd71-4dd4-bc67-595129b709a1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:38.637653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60cd46c0-cd71-4dd4-bc67-595129b709a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:38.653244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd46c0-cd71-4dd4-bc67-595129b709a1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:38.658990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:38.659787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:38.659957 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:38.660260 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] PATH_INFO: `/v3/projects/ee1da2c8aed14147a7e76d0e6b5d1108` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:38.663250 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1da2c8aed14147a7e76d0e6b5d1108)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:38.665190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:38.717344 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1da2c8aed14147a7e76d0e6b5d1108', 'request_id': 'req-8a908d20-eb95-4391-b890-fa5a856e74cf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.724148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1da2c8aed14147a7e76d0e6b5d1108', 'request_id': 'req-8a908d20-eb95-4391-b890-fa5a856e74cf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.729834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1da2c8aed14147a7e76d0e6b5d1108', 'request_id': 'req-8a908d20-eb95-4391-b890-fa5a856e74cf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.743130 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] The token cache is being invalidate because project ee1da2c8aed14147a7e76d0e6b5d1108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:38.743351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.743958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a908d20-eb95-4391-b890-fa5a856e74cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.745418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1193/2396] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:38 2022] DELETE /identity/v3/projects/ee1da2c8aed14147a7e76d0e6b5d1108 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:38.749441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0567e-3fbb-4b54-82c4-d7524b4785fe service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:38.760071 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0567e-3fbb-4b54-82c4-d7524b4785fe service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:38.778411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:38.822533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:38.823185 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:38.823466 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:38.823739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] PATH_INFO: `/v3/users/df562dbcd7ca42a18d03732d6ff81629` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:38.824864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df562dbcd7ca42a18d03732d6ff81629)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:38.826856 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:38.877360 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df562dbcd7ca42a18d03732d6ff81629'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.877917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df562dbcd7ca42a18d03732d6ff81629'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.883264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efc7e9-43ed-4396-adaa-b18923741548 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:38.883629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efc7e9-43ed-4396-adaa-b18923741548 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:38.883974 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efc7e9-43ed-4396-adaa-b18923741548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:38.891085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df562dbcd7ca42a18d03732d6ff81629'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.899000 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df562dbcd7ca42a18d03732d6ff81629'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.899715 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df562dbcd7ca42a18d03732d6ff81629'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:38.904144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-12efc7e9-43ed-4396-adaa-b18923741548 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:38.905351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12efc7e9-43ed-4396-adaa-b18923741548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:38.912435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1204/2397] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:38 2022] DELETE /identity/v3/users/df562dbcd7ca42a18d03732d6ff81629 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:38.966578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1194/2398] 10.222.0.16 () {58 vars in 967 bytes} [Mon Oct 17 10:35:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:38.980340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:38.981847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:39.056260 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ab5e-d7dd-4d4d-9314-bc8a5b82ff9e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:39.064012 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a08c62-e0b9-4d37-a5a5-cf8560c1708a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:39.064665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a08c62-e0b9-4d37-a5a5-cf8560c1708a service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.064913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a08c62-e0b9-4d37-a5a5-cf8560c1708a service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.065178 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a08c62-e0b9-4d37-a5a5-cf8560c1708a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.111088 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:39.128912 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a08c62-e0b9-4d37-a5a5-cf8560c1708a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:39.130552 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a08c62-e0b9-4d37-a5a5-cf8560c1708a service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:39.138841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1205/2399] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:35:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:35:39.195430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efc7e9-43ed-4396-adaa-b18923741548 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:39.197372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12efc7e9-43ed-4396-adaa-b18923741548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:39.217940 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efc7e9-43ed-4396-adaa-b18923741548 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:39.250659 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:39.286645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:39.287281 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.287556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.287827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] PATH_INFO: `/v3/projects/c1af72a1e44b40328ccc0a819bc5f283` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.288963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1af72a1e44b40328ccc0a819bc5f283)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:39.290944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:39.341418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1af72a1e44b40328ccc0a819bc5f283', 'request_id': 'req-bb838ada-c331-44e7-a655-fc360ba9ca87'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:39.348825 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1af72a1e44b40328ccc0a819bc5f283', 'request_id': 'req-bb838ada-c331-44e7-a655-fc360ba9ca87'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:39.355592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1af72a1e44b40328ccc0a819bc5f283', 'request_id': 'req-bb838ada-c331-44e7-a655-fc360ba9ca87'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:39.368419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] The token cache is being invalidate because project c1af72a1e44b40328ccc0a819bc5f283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:39.368711 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:39.369545 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bb838ada-c331-44e7-a655-fc360ba9ca87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:39.371090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1195/2400] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:39 2022] DELETE /identity/v3/projects/c1af72a1e44b40328ccc0a819bc5f283 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:39.376091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c1d06-828b-4fd7-8c1f-35dbb33a2faf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.376374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c1d06-828b-4fd7-8c1f-35dbb33a2faf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.376706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c1d06-828b-4fd7-8c1f-35dbb33a2faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.402076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-482c1d06-828b-4fd7-8c1f-35dbb33a2faf None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:39.403401 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-482c1d06-828b-4fd7-8c1f-35dbb33a2faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:39.468610 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1206/2401] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:35:39.533929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661f9d4-72fd-4382-864a-39d99f9f429e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.534243 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661f9d4-72fd-4382-864a-39d99f9f429e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.534601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661f9d4-72fd-4382-864a-39d99f9f429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.554961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d661f9d4-72fd-4382-864a-39d99f9f429e None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:39.556149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d661f9d4-72fd-4382-864a-39d99f9f429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:39.563961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1196/2402] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:39.568336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c1d06-828b-4fd7-8c1f-35dbb33a2faf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:39.579071 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c1d06-828b-4fd7-8c1f-35dbb33a2faf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:39.584416 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:39.585013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.585293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.585566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.586212 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:39.586551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:39.588154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:39.600749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1207/2403] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:35:39.605286 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661f9d4-72fd-4382-864a-39d99f9f429e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:39.614965 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661f9d4-72fd-4382-864a-39d99f9f429e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:39.619537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e6e6a-ec5f-42e1-8f81-29818d712917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:39.620119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e6e6a-ec5f-42e1-8f81-29818d712917 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.620390 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e6e6a-ec5f-42e1-8f81-29818d712917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.620656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e6e6a-ec5f-42e1-8f81-29818d712917 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.621308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e6e6a-ec5f-42e1-8f81-29818d712917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:39.623225 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e6e6a-ec5f-42e1-8f81-29818d712917 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:39.651921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1197/2404] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:39 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:39.656188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:39.667712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f458a7-b6b2-42e0-b43f-1077673a3a99 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:39.732778 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5190873-8a08-4d12-a81b-c3cdddee95e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:39.733387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5190873-8a08-4d12-a81b-c3cdddee95e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.733665 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5190873-8a08-4d12-a81b-c3cdddee95e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.733933 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5190873-8a08-4d12-a81b-c3cdddee95e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.735739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5190873-8a08-4d12-a81b-c3cdddee95e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:39.737148 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5190873-8a08-4d12-a81b-c3cdddee95e8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:39.762738 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1208/2405] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:39 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:39.766466 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e6e6a-ec5f-42e1-8f81-29818d712917 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:39.779033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e6e6a-ec5f-42e1-8f81-29818d712917 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:39.785301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e7111-865a-4e83-a9ac-4d92339b8a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:39.785903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7111-865a-4e83-a9ac-4d92339b8a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.786172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7111-865a-4e83-a9ac-4d92339b8a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.786512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7111-865a-4e83-a9ac-4d92339b8a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.787183 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e7111-865a-4e83-a9ac-4d92339b8a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:39.789682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e7111-865a-4e83-a9ac-4d92339b8a1b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:39.804139 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1198/2406] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:35:39.805036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5190873-8a08-4d12-a81b-c3cdddee95e8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:39.815695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5190873-8a08-4d12-a81b-c3cdddee95e8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:39.821201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d61d80-d79e-4681-b225-ce66ab01ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:39.821805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d61d80-d79e-4681-b225-ce66ab01ca2d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.822207 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d61d80-d79e-4681-b225-ce66ab01ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.822601 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d61d80-d79e-4681-b225-ce66ab01ca2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.823294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d61d80-d79e-4681-b225-ce66ab01ca2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:39.824972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d61d80-d79e-4681-b225-ce66ab01ca2d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:39.838914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1209/2407] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:39.843168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e7111-865a-4e83-a9ac-4d92339b8a1b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:39.853507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e7111-865a-4e83-a9ac-4d92339b8a1b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:39.862420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc4a9-d11d-4c3c-b028-3a6db7c05bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:39.863259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc4a9-d11d-4c3c-b028-3a6db7c05bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.863996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc4a9-d11d-4c3c-b028-3a6db7c05bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.863996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc4a9-d11d-4c3c-b028-3a6db7c05bb5 admin admin] PATH_INFO: `/v3/projects/2cc8e42eeec146ca86954f704fd81dc6/users/0f35a4b31ada473887d8c9eb0d1c52eb/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.881720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664bc4a9-d11d-4c3c-b028-3a6db7c05bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc8e42eeec146ca86954f704fd81dc6, user_id=0f35a4b31ada473887d8c9eb0d1c52eb, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:39.884431 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664bc4a9-d11d-4c3c-b028-3a6db7c05bb5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:39.898272 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1199/2408] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:39 2022] PUT /identity/v3/projects/2cc8e42eeec146ca86954f704fd81dc6/users/0f35a4b31ada473887d8c9eb0d1c52eb/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:39.908571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c86f6-aabd-460b-bb6f-05ad87c3031b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:39.908852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c86f6-aabd-460b-bb6f-05ad87c3031b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:39.909122 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c86f6-aabd-460b-bb6f-05ad87c3031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:39.935954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-109c86f6-aabd-460b-bb6f-05ad87c3031b None None] MFA Rules not processed for user `0f35a4b31ada473887d8c9eb0d1c52eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:39.937144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-109c86f6-aabd-460b-bb6f-05ad87c3031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:39.975571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:40.002387 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1210/2409] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:40.008180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc4a9-d11d-4c3c-b028-3a6db7c05bb5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:40.066528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:40.072419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc4a9-d11d-4c3c-b028-3a6db7c05bb5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:40.077891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bc7f8-b538-4da6-8b1b-d1d4559b6361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:40.078798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bc7f8-b538-4da6-8b1b-d1d4559b6361 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.079291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bc7f8-b538-4da6-8b1b-d1d4559b6361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.079557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bc7f8-b538-4da6-8b1b-d1d4559b6361 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.080441 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bc7f8-b538-4da6-8b1b-d1d4559b6361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:40.083266 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bc7f8-b538-4da6-8b1b-d1d4559b6361 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:40.109872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1200/2410] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:40 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:40.115090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c86f6-aabd-460b-bb6f-05ad87c3031b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:40.125530 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c86f6-aabd-460b-bb6f-05ad87c3031b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:40.182100 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95015f-cd80-4deb-b8ad-09780b8aab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:40.182725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95015f-cd80-4deb-b8ad-09780b8aab60 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.183021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95015f-cd80-4deb-b8ad-09780b8aab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.183265 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95015f-cd80-4deb-b8ad-09780b8aab60 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.183875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95015f-cd80-4deb-b8ad-09780b8aab60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:40.185826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95015f-cd80-4deb-b8ad-09780b8aab60 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:40.207638 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1211/2411] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:40 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:40.211597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bc7f8-b538-4da6-8b1b-d1d4559b6361 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:40.221558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bc7f8-b538-4da6-8b1b-d1d4559b6361 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:40.226372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ca5b-40a2-4c28-b77f-66246dabf846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:40.226961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ca5b-40a2-4c28-b77f-66246dabf846 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.227237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ca5b-40a2-4c28-b77f-66246dabf846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.227504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ca5b-40a2-4c28-b77f-66246dabf846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.228145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54ca5b-40a2-4c28-b77f-66246dabf846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:40.229720 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54ca5b-40a2-4c28-b77f-66246dabf846 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:40.240966 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1201/2412] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:40.245143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95015f-cd80-4deb-b8ad-09780b8aab60 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:40.255927 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95015f-cd80-4deb-b8ad-09780b8aab60 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:40.260622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3250e-58c2-493d-8294-78411c38937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:40.261370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3250e-58c2-493d-8294-78411c38937c admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.261653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3250e-58c2-493d-8294-78411c38937c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.261926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3250e-58c2-493d-8294-78411c38937c admin admin] PATH_INFO: `/v3/projects/a83c164338f34b219b184f66abdbd51c/users/9332e98c6b444c669691ea5530890b7a/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.276659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3250e-58c2-493d-8294-78411c38937c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83c164338f34b219b184f66abdbd51c, user_id=9332e98c6b444c669691ea5530890b7a, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:40.279347 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3250e-58c2-493d-8294-78411c38937c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:40.291592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1212/2413] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:40 2022] PUT /identity/v3/projects/a83c164338f34b219b184f66abdbd51c/users/9332e98c6b444c669691ea5530890b7a/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:40.295602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ca5b-40a2-4c28-b77f-66246dabf846 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:40.306041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ca5b-40a2-4c28-b77f-66246dabf846 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:40.364397 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d1e98-386f-46ca-a059-6ecf4385d273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:40.365607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d1e98-386f-46ca-a059-6ecf4385d273 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.365882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d1e98-386f-46ca-a059-6ecf4385d273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.366145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d1e98-386f-46ca-a059-6ecf4385d273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.366843 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d1e98-386f-46ca-a059-6ecf4385d273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:40.368406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d1e98-386f-46ca-a059-6ecf4385d273 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:40.378240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1202/2414] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:40.383169 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3250e-58c2-493d-8294-78411c38937c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:40.393382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3250e-58c2-493d-8294-78411c38937c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:40.398094 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:40.398765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.399040 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.399308 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] PATH_INFO: `/v3/projects/a83c164338f34b219b184f66abdbd51c/users/9332e98c6b444c669691ea5530890b7a/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.401214 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83c164338f34b219b184f66abdbd51c, user_id=9332e98c6b444c669691ea5530890b7a, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:40.403874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:40.414171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1213/2415] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:40 2022] PUT /identity/v3/projects/a83c164338f34b219b184f66abdbd51c/users/9332e98c6b444c669691ea5530890b7a/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:40.424308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610872e-3e7d-441d-8ea6-c5d770aa9fdf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.424585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610872e-3e7d-441d-8ea6-c5d770aa9fdf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.424875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610872e-3e7d-441d-8ea6-c5d770aa9fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.453106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7610872e-3e7d-441d-8ea6-c5d770aa9fdf None None] MFA Rules not processed for user `9332e98c6b444c669691ea5530890b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:40.454298 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7610872e-3e7d-441d-8ea6-c5d770aa9fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:40.521303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1203/2416] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:40.532613 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:40.534283 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:40.591599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c0dde-6057-4bee-b5b6-46c7ae2446da admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:40.595176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1ae0b-4c1d-40b3-9e86-aedc796c263c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:40.595757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ae0b-4c1d-40b3-9e86-aedc796c263c service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.596036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ae0b-4c1d-40b3-9e86-aedc796c263c service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.596294 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ae0b-4c1d-40b3-9e86-aedc796c263c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.606075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1ae0b-4c1d-40b3-9e86-aedc796c263c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:40.607534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1ae0b-4c1d-40b3-9e86-aedc796c263c service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:40.617627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1214/2417] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:40.691083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610872e-3e7d-441d-8ea6-c5d770aa9fdf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:40.702845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610872e-3e7d-441d-8ea6-c5d770aa9fdf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:40.706446 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ec176-bd5a-4b39-8052-d05d981f007f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:40.707088 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ec176-bd5a-4b39-8052-d05d981f007f service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:40.707418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ec176-bd5a-4b39-8052-d05d981f007f service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:40.707626 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ec176-bd5a-4b39-8052-d05d981f007f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:40.756135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:40.766177 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ec176-bd5a-4b39-8052-d05d981f007f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:40.767612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ec176-bd5a-4b39-8052-d05d981f007f service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:40.776715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1204/2418] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:41.059783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1ae0b-4c1d-40b3-9e86-aedc796c263c service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:41.100459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:41.130441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1ae0b-4c1d-40b3-9e86-aedc796c263c service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:41.134842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:41.135464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:41.135735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:41.136005 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] PATH_INFO: `/v3/users/0f35a4b31ada473887d8c9eb0d1c52eb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:41.137105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f35a4b31ada473887d8c9eb0d1c52eb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:41.139082 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:41.185201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f35a4b31ada473887d8c9eb0d1c52eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.185833 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f35a4b31ada473887d8c9eb0d1c52eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.197761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f35a4b31ada473887d8c9eb0d1c52eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.204911 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f35a4b31ada473887d8c9eb0d1c52eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.205531 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f35a4b31ada473887d8c9eb0d1c52eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.220143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1215/2419] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:41 2022] DELETE /identity/v3/users/0f35a4b31ada473887d8c9eb0d1c52eb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:41.225213 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09161-e342-4184-b6ef-e15ac47200ca None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:41.225689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09161-e342-4184-b6ef-e15ac47200ca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:41.225884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09161-e342-4184-b6ef-e15ac47200ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:41.245621 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-86b09161-e342-4184-b6ef-e15ac47200ca None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:41.246977 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86b09161-e342-4184-b6ef-e15ac47200ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:41.306620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1205/2420] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:41.317253 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:41.318682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:41.375171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e25ac2-fac6-4fcc-879e-acc2ffc64513 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:41.379636 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28623bce-a9fa-44a1-8a94-86ac4bf1b279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:41.380226 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28623bce-a9fa-44a1-8a94-86ac4bf1b279 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:41.380550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28623bce-a9fa-44a1-8a94-86ac4bf1b279 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:41.380764 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28623bce-a9fa-44a1-8a94-86ac4bf1b279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:41.389428 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28623bce-a9fa-44a1-8a94-86ac4bf1b279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:41.390901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28623bce-a9fa-44a1-8a94-86ac4bf1b279 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:41.401237 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1216/2421] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:41.671590 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09161-e342-4184-b6ef-e15ac47200ca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:41.673033 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86b09161-e342-4184-b6ef-e15ac47200ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:41.687128 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09161-e342-4184-b6ef-e15ac47200ca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:41.692631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:41.693246 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:41.693517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:41.693791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] PATH_INFO: `/v3/projects/2cc8e42eeec146ca86954f704fd81dc6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:41.695594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc8e42eeec146ca86954f704fd81dc6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:41.697497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:41.744617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc8e42eeec146ca86954f704fd81dc6', 'request_id': 'req-d701cea9-71a6-46cd-81a7-c6b33ff0402e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.750695 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc8e42eeec146ca86954f704fd81dc6', 'request_id': 'req-d701cea9-71a6-46cd-81a7-c6b33ff0402e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.756883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc8e42eeec146ca86954f704fd81dc6', 'request_id': 'req-d701cea9-71a6-46cd-81a7-c6b33ff0402e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.768675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] The token cache is being invalidate because project 2cc8e42eeec146ca86954f704fd81dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:41.768963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.770034 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.771528 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1206/2422] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:41 2022] DELETE /identity/v3/projects/2cc8e42eeec146ca86954f704fd81dc6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:41.775558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28623bce-a9fa-44a1-8a94-86ac4bf1b279 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:41.784944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28623bce-a9fa-44a1-8a94-86ac4bf1b279 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:41.835749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:41.844267 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:41.844906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:41.845368 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:41.845830 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] PATH_INFO: `/v3/users/9332e98c6b444c669691ea5530890b7a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:41.846990 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9332e98c6b444c669691ea5530890b7a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:41.849096 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:41.896772 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9332e98c6b444c669691ea5530890b7a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.897418 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9332e98c6b444c669691ea5530890b7a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.909016 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9332e98c6b444c669691ea5530890b7a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.916213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9332e98c6b444c669691ea5530890b7a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.916758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06fd39f3-3f90-499a-b8f4-3233284b7f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9332e98c6b444c669691ea5530890b7a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:41.929251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1217/2423] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:35:41 2022] DELETE /identity/v3/users/9332e98c6b444c669691ea5530890b7a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:35:42.203279 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:42.204730 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:42.219204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701cea9-71a6-46cd-81a7-c6b33ff0402e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:42.278638 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:42.279309 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.279524 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.279792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] PATH_INFO: `/v3/projects/a83c164338f34b219b184f66abdbd51c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.280895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a83c164338f34b219b184f66abdbd51c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:42.282829 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:42.298078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:42.330258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a83c164338f34b219b184f66abdbd51c', 'request_id': 'req-c2f8040e-5b8b-4861-b810-415394ad8bf3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:42.336368 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a83c164338f34b219b184f66abdbd51c', 'request_id': 'req-c2f8040e-5b8b-4861-b810-415394ad8bf3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:42.341786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a83c164338f34b219b184f66abdbd51c', 'request_id': 'req-c2f8040e-5b8b-4861-b810-415394ad8bf3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:42.353975 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] The token cache is being invalidate because project a83c164338f34b219b184f66abdbd51c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:35:42.354278 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:42.354944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2f8040e-5b8b-4861-b810-415394ad8bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:35:42.356684 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1207/2424] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:35:42 2022] DELETE /identity/v3/projects/a83c164338f34b219b184f66abdbd51c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:42.361572 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4d6bc-4809-4504-b3d8-56d4f3d69dc6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.362060 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4d6bc-4809-4504-b3d8-56d4f3d69dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.362158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4d6bc-4809-4504-b3d8-56d4f3d69dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.382850 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c4a4d6bc-4809-4504-b3d8-56d4f3d69dc6 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:42.384023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c4a4d6bc-4809-4504-b3d8-56d4f3d69dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:42.441619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1218/2425] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:42.450027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae42c2-355b-4278-8e43-ea8d1efd4e84 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.450251 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae42c2-355b-4278-8e43-ea8d1efd4e84 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.450557 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae42c2-355b-4278-8e43-ea8d1efd4e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.470206 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8fae42c2-355b-4278-8e43-ea8d1efd4e84 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:42.471427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8fae42c2-355b-4278-8e43-ea8d1efd4e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:42.479211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1208/2426] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:42.483493 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4d6bc-4809-4504-b3d8-56d4f3d69dc6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:42.493564 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4d6bc-4809-4504-b3d8-56d4f3d69dc6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:42.498101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:42.498726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.499003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.499271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.499910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:35:42.500215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:42.501796 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:42.513756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1219/2427] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:35:42.518061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae42c2-355b-4278-8e43-ea8d1efd4e84 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:42.527321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae42c2-355b-4278-8e43-ea8d1efd4e84 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:42.531945 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0c40-8d37-4686-9a43-616ab0446c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:42.532530 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0c40-8d37-4686-9a43-616ab0446c46 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.532804 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0c40-8d37-4686-9a43-616ab0446c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.533073 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0c40-8d37-4686-9a43-616ab0446c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.533727 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe0c40-8d37-4686-9a43-616ab0446c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:42.535645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe0c40-8d37-4686-9a43-616ab0446c46 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:42.562909 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1209/2428] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:35:42 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:42.567137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:42.577382 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28243626-f96e-4104-8182-3bb07b51d24a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:42.632745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec33e8c-5644-4bde-a2aa-098f96d9f640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:42.633342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33e8c-5644-4bde-a2aa-098f96d9f640 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.633618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33e8c-5644-4bde-a2aa-098f96d9f640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.633886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33e8c-5644-4bde-a2aa-098f96d9f640 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.634555 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec33e8c-5644-4bde-a2aa-098f96d9f640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:42.636469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec33e8c-5644-4bde-a2aa-098f96d9f640 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:42.645303 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:35:42.658963 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1220/2429] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:35:42 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:35:42.663584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0c40-8d37-4686-9a43-616ab0446c46 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:42.673609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0c40-8d37-4686-9a43-616ab0446c46 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:42.679071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7755380-caeb-404b-8ba5-ab44788bbf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:42.679660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755380-caeb-404b-8ba5-ab44788bbf90 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.679953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755380-caeb-404b-8ba5-ab44788bbf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.680204 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755380-caeb-404b-8ba5-ab44788bbf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.680840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7755380-caeb-404b-8ba5-ab44788bbf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:42.682454 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7755380-caeb-404b-8ba5-ab44788bbf90 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:42.692363 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1210/2430] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:42.697163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec33e8c-5644-4bde-a2aa-098f96d9f640 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:42.706851 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec33e8c-5644-4bde-a2aa-098f96d9f640 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:42.711442 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34924975-4b40-487b-9f27-c4a235a18283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:42.712023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34924975-4b40-487b-9f27-c4a235a18283 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.712310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34924975-4b40-487b-9f27-c4a235a18283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.712561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34924975-4b40-487b-9f27-c4a235a18283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.713196 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34924975-4b40-487b-9f27-c4a235a18283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:42.714832 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34924975-4b40-487b-9f27-c4a235a18283 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:42.726734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1221/2431] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:35:42.731134 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7755380-caeb-404b-8ba5-ab44788bbf90 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:42.742258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7755380-caeb-404b-8ba5-ab44788bbf90 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:42.746845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:42.747477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.747746 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.748011 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] PATH_INFO: `/v3/projects/6cbfcec8fd0b42c2b66641b5238083e9/users/7e9b10419f834e0a80d1f0a6023702c2/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.762147 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbfcec8fd0b42c2b66641b5238083e9, user_id=7e9b10419f834e0a80d1f0a6023702c2, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:42.764781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:42.778171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1211/2432] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:35:42 2022] PUT /identity/v3/projects/6cbfcec8fd0b42c2b66641b5238083e9/users/7e9b10419f834e0a80d1f0a6023702c2/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:35:42.788925 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079903c-405c-4163-ae32-697cd4cdc4b5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.789206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079903c-405c-4163-ae32-697cd4cdc4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.789475 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079903c-405c-4163-ae32-697cd4cdc4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.817014 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6079903c-405c-4163-ae32-697cd4cdc4b5 None None] MFA Rules not processed for user `7e9b10419f834e0a80d1f0a6023702c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:35:42.818251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6079903c-405c-4163-ae32-697cd4cdc4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:42.883599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1222/2433] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:35:42.894873 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:42.896715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:42.953323 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a9ca5-529a-4b24-983f-7be514a37643 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:42.956999 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:42.957532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:42.957808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:42.958220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:42.966451 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:42.967857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:42.977010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1212/2434] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:52.632827 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079903c-405c-4163-ae32-697cd4cdc4b5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:35:52.634634 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6079903c-405c-4163-ae32-697cd4cdc4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:52.707931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079903c-405c-4163-ae32-697cd4cdc4b5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:35:52.713891 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd87ba-3d4d-4716-9574-abb9313f331d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:35:52.714525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd87ba-3d4d-4716-9574-abb9313f331d service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:35:52.714798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd87ba-3d4d-4716-9574-abb9313f331d service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:35:52.715069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd87ba-3d4d-4716-9574-abb9313f331d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:35:52.716879 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-efcd87ba-3d4d-4716-9574-abb9313f331d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:35:52.730437 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd87ba-3d4d-4716-9574-abb9313f331d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:35:52.732027 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd87ba-3d4d-4716-9574-abb9313f331d service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:35:52.742998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1223/2435] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:35:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:35:52.972525 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:00.279745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:00.282280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:00.354754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710efd7-6985-40e0-acdb-cc7e21c5c2d2 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:00.362128 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:00.362799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:00.363077 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:00.363349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] PATH_INFO: `/v3/users/29f490bb81f640a4b2ff7e1d16313151` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:00.364548 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f490bb81f640a4b2ff7e1d16313151)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:00.366866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:00.416520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f490bb81f640a4b2ff7e1d16313151'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:00.417329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f490bb81f640a4b2ff7e1d16313151'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:00.425401 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f490bb81f640a4b2ff7e1d16313151'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:00.438027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f490bb81f640a4b2ff7e1d16313151'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:00.438732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f490bb81f640a4b2ff7e1d16313151'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:00.451735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1213/2436] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:36:00 2022] DELETE /identity/v3/users/29f490bb81f640a4b2ff7e1d16313151 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:00.456525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370326c-e2e6-48b2-b5d7-1c3a01972bd2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:00.457050 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370326c-e2e6-48b2-b5d7-1c3a01972bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:00.457164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370326c-e2e6-48b2-b5d7-1c3a01972bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:00.477901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9370326c-e2e6-48b2-b5d7-1c3a01972bd2 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:00.479183 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9370326c-e2e6-48b2-b5d7-1c3a01972bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:00.531254 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:00.546981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1224/2437] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:36:00.557086 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:00.558593 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:00.628838 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a14e6-ba7a-4777-ad3b-477d23d272fd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:00.633712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f48ab-a6e7-4a3b-92bc-1a4308151d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:00.634428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f48ab-a6e7-4a3b-92bc-1a4308151d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:00.634765 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f48ab-a6e7-4a3b-92bc-1a4308151d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:00.635097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f48ab-a6e7-4a3b-92bc-1a4308151d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:00.647174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f48ab-a6e7-4a3b-92bc-1a4308151d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:00.648667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f48ab-a6e7-4a3b-92bc-1a4308151d30 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:00.667925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1214/2438] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:00.935771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370326c-e2e6-48b2-b5d7-1c3a01972bd2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:00.937273 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9370326c-e2e6-48b2-b5d7-1c3a01972bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:00.950621 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370326c-e2e6-48b2-b5d7-1c3a01972bd2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:00.955455 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:00.956089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:00.956366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:00.956687 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] PATH_INFO: `/v3/projects/d7ec7ea91361407b9d2550cb13ade802` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:00.957854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ec7ea91361407b9d2550cb13ade802)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:00.960767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.007330 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ec7ea91361407b9d2550cb13ade802', 'request_id': 'req-b5f1cfe7-2451-451c-9add-b8cc20220f22'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:01.013204 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ec7ea91361407b9d2550cb13ade802', 'request_id': 'req-b5f1cfe7-2451-451c-9add-b8cc20220f22'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:01.020173 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ec7ea91361407b9d2550cb13ade802', 'request_id': 'req-b5f1cfe7-2451-451c-9add-b8cc20220f22'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:01.031224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] The token cache is being invalidate because project d7ec7ea91361407b9d2550cb13ade802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:36:01.031507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:01.032117 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b5f1cfe7-2451-451c-9add-b8cc20220f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:01.033569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1225/2439] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:36:00 2022] DELETE /identity/v3/projects/d7ec7ea91361407b9d2550cb13ade802 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:01.039190 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9f35-f6e5-4b4a-89b8-ed043536fafe None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.039473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9f35-f6e5-4b4a-89b8-ed043536fafe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.039744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9f35-f6e5-4b4a-89b8-ed043536fafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.059220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fcda9f35-f6e5-4b4a-89b8-ed043536fafe None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:01.060408 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fcda9f35-f6e5-4b4a-89b8-ed043536fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:01.077423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:01.122392 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1215/2440] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:01.130236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f222fe-00de-4fac-b7cb-3d616b22a19b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.130594 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f222fe-00de-4fac-b7cb-3d616b22a19b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.130882 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f222fe-00de-4fac-b7cb-3d616b22a19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.151438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-10f222fe-00de-4fac-b7cb-3d616b22a19b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:01.152632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-10f222fe-00de-4fac-b7cb-3d616b22a19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:01.160366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1226/2441] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:01.165062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9f35-f6e5-4b4a-89b8-ed043536fafe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.174662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9f35-f6e5-4b4a-89b8-ed043536fafe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.179214 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.179798 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.180075 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.180343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.180986 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:36:01.181285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.182896 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.194143 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1216/2442] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:36:01.198235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f222fe-00de-4fac-b7cb-3d616b22a19b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.207160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f222fe-00de-4fac-b7cb-3d616b22a19b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.212332 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e740ee-bd06-4413-a7c1-f18e359dc28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.212917 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e740ee-bd06-4413-a7c1-f18e359dc28d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.213191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e740ee-bd06-4413-a7c1-f18e359dc28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.213459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e740ee-bd06-4413-a7c1-f18e359dc28d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.214089 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e740ee-bd06-4413-a7c1-f18e359dc28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.216013 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e740ee-bd06-4413-a7c1-f18e359dc28d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.244632 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1227/2443] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:36:01 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:36:01.248786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.258893 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e842b88-8e25-488b-ac03-9dc6192569cd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.317600 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfea4d-30ae-4478-89e3-6a44110d3399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.318203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfea4d-30ae-4478-89e3-6a44110d3399 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.318537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfea4d-30ae-4478-89e3-6a44110d3399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.318831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfea4d-30ae-4478-89e3-6a44110d3399 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.319654 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfea4d-30ae-4478-89e3-6a44110d3399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.321558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfea4d-30ae-4478-89e3-6a44110d3399 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.344475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1217/2444] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:36:01 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:36:01.348751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e740ee-bd06-4413-a7c1-f18e359dc28d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.360695 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e740ee-bd06-4413-a7c1-f18e359dc28d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.367280 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873d2e-2f3e-4bd9-9c16-775bfddf9be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.367884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873d2e-2f3e-4bd9-9c16-775bfddf9be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.368162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873d2e-2f3e-4bd9-9c16-775bfddf9be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.368430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873d2e-2f3e-4bd9-9c16-775bfddf9be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.369070 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f873d2e-2f3e-4bd9-9c16-775bfddf9be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.370741 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f873d2e-2f3e-4bd9-9c16-775bfddf9be5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.382450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1228/2445] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:36:01.386958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfea4d-30ae-4478-89e3-6a44110d3399 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.396623 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfea4d-30ae-4478-89e3-6a44110d3399 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.401983 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aae77d-b184-4a59-840f-8402922f0a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.403065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae77d-b184-4a59-840f-8402922f0a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.403335 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae77d-b184-4a59-840f-8402922f0a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.403601 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae77d-b184-4a59-840f-8402922f0a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.404236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aae77d-b184-4a59-840f-8402922f0a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.405792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aae77d-b184-4a59-840f-8402922f0a74 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.417153 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1218/2446] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:36:01.421338 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873d2e-2f3e-4bd9-9c16-775bfddf9be5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.431554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873d2e-2f3e-4bd9-9c16-775bfddf9be5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.436107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24693b-ed87-4a33-99fe-89404a0bc3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.436799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24693b-ed87-4a33-99fe-89404a0bc3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.437074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24693b-ed87-4a33-99fe-89404a0bc3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.437340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24693b-ed87-4a33-99fe-89404a0bc3a4 admin admin] PATH_INFO: `/v3/projects/b989ca83aecd4a629e66ff3f902054fb/users/575b55954aac43eea0d9c52a40fd3749/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.452569 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24693b-ed87-4a33-99fe-89404a0bc3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b989ca83aecd4a629e66ff3f902054fb, user_id=575b55954aac43eea0d9c52a40fd3749, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.455236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24693b-ed87-4a33-99fe-89404a0bc3a4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.468130 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1229/2447] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:36:01 2022] PUT /identity/v3/projects/b989ca83aecd4a629e66ff3f902054fb/users/575b55954aac43eea0d9c52a40fd3749/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:01.479841 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8bbd3-f0de-4b6b-b935-56742c1764b3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.480490 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8bbd3-f0de-4b6b-b935-56742c1764b3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.480789 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8bbd3-f0de-4b6b-b935-56742c1764b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.509283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-90e8bbd3-f0de-4b6b-b935-56742c1764b3 None None] MFA Rules not processed for user `575b55954aac43eea0d9c52a40fd3749`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:01.510489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-90e8bbd3-f0de-4b6b-b935-56742c1764b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:01.574681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1219/2448] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:01.579287 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24693b-ed87-4a33-99fe-89404a0bc3a4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.640779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24693b-ed87-4a33-99fe-89404a0bc3a4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.645500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a873be-ea9f-4fec-bbb5-bbedb2602445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.646090 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a873be-ea9f-4fec-bbb5-bbedb2602445 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.646392 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a873be-ea9f-4fec-bbb5-bbedb2602445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.647609 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a873be-ea9f-4fec-bbb5-bbedb2602445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.648235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a873be-ea9f-4fec-bbb5-bbedb2602445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.650119 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a873be-ea9f-4fec-bbb5-bbedb2602445 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.670735 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:01.678637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1230/2449] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:36:01 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:36:01.682722 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8bbd3-f0de-4b6b-b935-56742c1764b3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.693396 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8bbd3-f0de-4b6b-b935-56742c1764b3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.706586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:01.752773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd431f1-5cf2-4c3e-835a-c3264cc3cf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.753369 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd431f1-5cf2-4c3e-835a-c3264cc3cf04 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.753649 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd431f1-5cf2-4c3e-835a-c3264cc3cf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.753915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd431f1-5cf2-4c3e-835a-c3264cc3cf04 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.754574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd431f1-5cf2-4c3e-835a-c3264cc3cf04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.757169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd431f1-5cf2-4c3e-835a-c3264cc3cf04 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.780881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1220/2450] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:36:01 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:36:01.784894 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a873be-ea9f-4fec-bbb5-bbedb2602445 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.794972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a873be-ea9f-4fec-bbb5-bbedb2602445 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.799500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16113724-eeb2-444c-b324-508c3c40f974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.800076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113724-eeb2-444c-b324-508c3c40f974 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.800349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113724-eeb2-444c-b324-508c3c40f974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.800620 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113724-eeb2-444c-b324-508c3c40f974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.801247 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16113724-eeb2-444c-b324-508c3c40f974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.803374 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16113724-eeb2-444c-b324-508c3c40f974 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.814886 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1231/2451] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:36:01.819899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd431f1-5cf2-4c3e-835a-c3264cc3cf04 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.831610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd431f1-5cf2-4c3e-835a-c3264cc3cf04 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.836268 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed499c-c9cb-4bd6-b7fb-38cef6effc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.836912 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed499c-c9cb-4bd6-b7fb-38cef6effc8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.837184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed499c-c9cb-4bd6-b7fb-38cef6effc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.837457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed499c-c9cb-4bd6-b7fb-38cef6effc8f admin admin] PATH_INFO: `/v3/projects/c1d099aa8f8c4b53911d0b8de8ca259c/users/96b9b14effeb444f819d2e4e0900bc14/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.854291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed499c-c9cb-4bd6-b7fb-38cef6effc8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d099aa8f8c4b53911d0b8de8ca259c, user_id=96b9b14effeb444f819d2e4e0900bc14, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.857497 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed499c-c9cb-4bd6-b7fb-38cef6effc8f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.873068 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1221/2452] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:36:01 2022] PUT /identity/v3/projects/c1d099aa8f8c4b53911d0b8de8ca259c/users/96b9b14effeb444f819d2e4e0900bc14/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:36:01.877696 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16113724-eeb2-444c-b324-508c3c40f974 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.890470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16113724-eeb2-444c-b324-508c3c40f974 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.955315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981c1b7-bd98-4cb5-833b-602406671999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.955919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981c1b7-bd98-4cb5-833b-602406671999 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.956198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981c1b7-bd98-4cb5-833b-602406671999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.956469 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981c1b7-bd98-4cb5-833b-602406671999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.957118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981c1b7-bd98-4cb5-833b-602406671999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.958736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981c1b7-bd98-4cb5-833b-602406671999 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:01.968611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1232/2453] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:36:01.972967 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed499c-c9cb-4bd6-b7fb-38cef6effc8f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:01.983956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed499c-c9cb-4bd6-b7fb-38cef6effc8f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:01.988537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:01.989174 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:01.989452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:01.989726 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] PATH_INFO: `/v3/projects/c1d099aa8f8c4b53911d0b8de8ca259c/users/96b9b14effeb444f819d2e4e0900bc14/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:01.992165 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d099aa8f8c4b53911d0b8de8ca259c, user_id=96b9b14effeb444f819d2e4e0900bc14, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:01.994832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:02.004827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1222/2454] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:36:01 2022] PUT /identity/v3/projects/c1d099aa8f8c4b53911d0b8de8ca259c/users/96b9b14effeb444f819d2e4e0900bc14/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:02.015793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8ae-27a8-44ed-a9dc-e8772cf25425 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:02.016085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8ae-27a8-44ed-a9dc-e8772cf25425 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:02.016358 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8ae-27a8-44ed-a9dc-e8772cf25425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:02.043920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-01e1f8ae-27a8-44ed-a9dc-e8772cf25425 None None] MFA Rules not processed for user `96b9b14effeb444f819d2e4e0900bc14`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:02.045378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-01e1f8ae-27a8-44ed-a9dc-e8772cf25425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:02.113955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1233/2455] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:02.125892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:02.127666 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:02.185349 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fbd4f-ec27-463d-bd03-a194b7ad287e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:02.188870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59b4c3-392b-401e-b77b-adc44448dbe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:02.189459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59b4c3-392b-401e-b77b-adc44448dbe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:02.189731 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59b4c3-392b-401e-b77b-adc44448dbe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:02.189998 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59b4c3-392b-401e-b77b-adc44448dbe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:02.245181 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59b4c3-392b-401e-b77b-adc44448dbe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:02.246648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59b4c3-392b-401e-b77b-adc44448dbe9 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:02.255385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1223/2456] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:07.980075 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1f8ae-27a8-44ed-a9dc-e8772cf25425 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:08.042628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1f8ae-27a8-44ed-a9dc-e8772cf25425 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:08.047140 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:08.047771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:08.048046 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:08.048318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] PATH_INFO: `/v3/users/575b55954aac43eea0d9c52a40fd3749` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:08.049453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] RBAC: Authorizing `identity:delete_user(user_id=575b55954aac43eea0d9c52a40fd3749)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:08.051404 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:08.066451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:08.106115 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '575b55954aac43eea0d9c52a40fd3749'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.106901 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '575b55954aac43eea0d9c52a40fd3749'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.120002 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '575b55954aac43eea0d9c52a40fd3749'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.127975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '575b55954aac43eea0d9c52a40fd3749'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.128666 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '575b55954aac43eea0d9c52a40fd3749'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.143227 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1234/2457] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:36:07 2022] DELETE /identity/v3/users/575b55954aac43eea0d9c52a40fd3749 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:08.149295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf77bf-2731-4bb7-853c-8409e6944e14 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:08.149872 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf77bf-2731-4bb7-853c-8409e6944e14 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:08.149982 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf77bf-2731-4bb7-853c-8409e6944e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:08.175140 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-41cf77bf-2731-4bb7-853c-8409e6944e14 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:08.176964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-41cf77bf-2731-4bb7-853c-8409e6944e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:08.198854 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:08.245771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1224/2458] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:36:08.261305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:08.262797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:08.326179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735741ed-db95-40ac-a8c8-8475c1fda08e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:08.329857 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cabd17-9417-46ea-9e70-07c1f87c8392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:08.330573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cabd17-9417-46ea-9e70-07c1f87c8392 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:08.330870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cabd17-9417-46ea-9e70-07c1f87c8392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:08.331251 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cabd17-9417-46ea-9e70-07c1f87c8392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:08.341611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cabd17-9417-46ea-9e70-07c1f87c8392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:08.343091 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cabd17-9417-46ea-9e70-07c1f87c8392 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:08.354398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1235/2459] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:08.660657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf77bf-2731-4bb7-853c-8409e6944e14 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:08.662080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-41cf77bf-2731-4bb7-853c-8409e6944e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:08.683079 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf77bf-2731-4bb7-853c-8409e6944e14 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:08.699168 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:08.699930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:08.700377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:08.700507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] PATH_INFO: `/v3/projects/b989ca83aecd4a629e66ff3f902054fb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:08.701681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b989ca83aecd4a629e66ff3f902054fb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:08.706060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:08.773478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b989ca83aecd4a629e66ff3f902054fb', 'request_id': 'req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.786003 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b989ca83aecd4a629e66ff3f902054fb', 'request_id': 'req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.795468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b989ca83aecd4a629e66ff3f902054fb', 'request_id': 'req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.814144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] The token cache is being invalidate because project b989ca83aecd4a629e66ff3f902054fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:36:08.814495 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.815405 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.818419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1225/2460] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:36:08 2022] DELETE /identity/v3/projects/b989ca83aecd4a629e66ff3f902054fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:08.823375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cabd17-9417-46ea-9e70-07c1f87c8392 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:08.833314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cabd17-9417-46ea-9e70-07c1f87c8392 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:08.897608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:08.898163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:08.898470 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:08.898736 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] PATH_INFO: `/v3/users/96b9b14effeb444f819d2e4e0900bc14` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:08.900344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b9b14effeb444f819d2e4e0900bc14)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:08.902262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:08.939001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:08.951243 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b9b14effeb444f819d2e4e0900bc14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.951889 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b9b14effeb444f819d2e4e0900bc14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.963794 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b9b14effeb444f819d2e4e0900bc14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.970676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b9b14effeb444f819d2e4e0900bc14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.971281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d15b926-370e-4d91-b37f-0cf1f2666e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b9b14effeb444f819d2e4e0900bc14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:08.983935 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1236/2461] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:36:08 2022] DELETE /identity/v3/users/96b9b14effeb444f819d2e4e0900bc14 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:09.264457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:09.266550 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:09.281535 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8ec7d-8a94-4ea1-abaa-50d3215d292e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:09.343541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:09.344163 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.344437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.344704 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] PATH_INFO: `/v3/projects/c1d099aa8f8c4b53911d0b8de8ca259c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.345790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d099aa8f8c4b53911d0b8de8ca259c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:09.347701 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:09.396457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d099aa8f8c4b53911d0b8de8ca259c', 'request_id': 'req-a796c806-9b14-41dc-a700-d67a3fa410f3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:09.397572 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:09.405038 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d099aa8f8c4b53911d0b8de8ca259c', 'request_id': 'req-a796c806-9b14-41dc-a700-d67a3fa410f3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:09.411232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d099aa8f8c4b53911d0b8de8ca259c', 'request_id': 'req-a796c806-9b14-41dc-a700-d67a3fa410f3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:09.423779 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] The token cache is being invalidate because project c1d099aa8f8c4b53911d0b8de8ca259c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:36:09.423981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:09.424582 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a796c806-9b14-41dc-a700-d67a3fa410f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:09.426025 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1226/2462] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:36:09 2022] DELETE /identity/v3/projects/c1d099aa8f8c4b53911d0b8de8ca259c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:09.431120 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8eb3a-90b8-4d47-950f-c416e61383df None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.431414 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8eb3a-90b8-4d47-950f-c416e61383df None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.431688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8eb3a-90b8-4d47-950f-c416e61383df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.450998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-dbf8eb3a-90b8-4d47-950f-c416e61383df None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:09.452163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dbf8eb3a-90b8-4d47-950f-c416e61383df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:09.509819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1237/2463] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:09.518565 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a56a-5a27-4eef-a6d9-c696868af35a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.518848 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a56a-5a27-4eef-a6d9-c696868af35a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.519135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a56a-5a27-4eef-a6d9-c696868af35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.541006 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b9a2a56a-5a27-4eef-a6d9-c696868af35a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:09.542189 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b9a2a56a-5a27-4eef-a6d9-c696868af35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:09.552022 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1227/2464] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:09.556628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8eb3a-90b8-4d47-950f-c416e61383df None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:09.566019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8eb3a-90b8-4d47-950f-c416e61383df None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:09.570529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:09.571639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.571914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.572179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.572847 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:36:09.573147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:09.574760 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:09.587424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1238/2465] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:36:09.591561 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2a56a-5a27-4eef-a6d9-c696868af35a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:09.600792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2a56a-5a27-4eef-a6d9-c696868af35a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:09.605889 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817fe5d-d54f-45f0-b725-2837a5d0afbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:09.606610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817fe5d-d54f-45f0-b725-2837a5d0afbf admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.606946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817fe5d-d54f-45f0-b725-2837a5d0afbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.607271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817fe5d-d54f-45f0-b725-2837a5d0afbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.607956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817fe5d-d54f-45f0-b725-2837a5d0afbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:09.610751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817fe5d-d54f-45f0-b725-2837a5d0afbf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:09.650464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1228/2466] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:36:09 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:36:09.654431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:09.664200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27357fad-6b8f-4f10-b0ea-bf9432471a13 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:09.724864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e12ff-97b3-4e2d-bffc-be3d192de1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:09.725471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e12ff-97b3-4e2d-bffc-be3d192de1aa admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.725876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e12ff-97b3-4e2d-bffc-be3d192de1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.726150 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e12ff-97b3-4e2d-bffc-be3d192de1aa admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.726818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e12ff-97b3-4e2d-bffc-be3d192de1aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:09.728787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e12ff-97b3-4e2d-bffc-be3d192de1aa admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:09.753380 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1239/2467] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:36:09 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:36:09.754717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817fe5d-d54f-45f0-b725-2837a5d0afbf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:09.765772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817fe5d-d54f-45f0-b725-2837a5d0afbf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:09.770793 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ed4ee-ef52-420f-a57b-f2b0e76fe3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:09.771625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed4ee-ef52-420f-a57b-f2b0e76fe3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.771625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed4ee-ef52-420f-a57b-f2b0e76fe3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.771939 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed4ee-ef52-420f-a57b-f2b0e76fe3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.772633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ed4ee-ef52-420f-a57b-f2b0e76fe3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:09.774790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ed4ee-ef52-420f-a57b-f2b0e76fe3d1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:09.785373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1229/2468] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:36:09.789627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e12ff-97b3-4e2d-bffc-be3d192de1aa admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:09.800726 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e12ff-97b3-4e2d-bffc-be3d192de1aa admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:09.805343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb974d-d2de-4adf-b22a-5ff811b8d893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:09.806400 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb974d-d2de-4adf-b22a-5ff811b8d893 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.806680 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb974d-d2de-4adf-b22a-5ff811b8d893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.806947 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb974d-d2de-4adf-b22a-5ff811b8d893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.807582 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb974d-d2de-4adf-b22a-5ff811b8d893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:09.809162 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb974d-d2de-4adf-b22a-5ff811b8d893 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:09.819981 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1240/2469] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:36:09.824729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ed4ee-ef52-420f-a57b-f2b0e76fe3d1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:09.834826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ed4ee-ef52-420f-a57b-f2b0e76fe3d1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:09.840071 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:09.840719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.841000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.841270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] PATH_INFO: `/v3/projects/711ab39e17604b0d9df8ef228108396c/users/90e4dd806c804309aece8b368e0fb67e/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.856385 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=711ab39e17604b0d9df8ef228108396c, user_id=90e4dd806c804309aece8b368e0fb67e, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:09.859127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:09.873542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1230/2470] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:36:09 2022] PUT /identity/v3/projects/711ab39e17604b0d9df8ef228108396c/users/90e4dd806c804309aece8b368e0fb67e/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:09.878676 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5686adb-f120-4f74-9e47-4709cb6a6839 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:09.878955 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5686adb-f120-4f74-9e47-4709cb6a6839 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:09.879221 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5686adb-f120-4f74-9e47-4709cb6a6839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:09.900472 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a5686adb-f120-4f74-9e47-4709cb6a6839 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:09.901724 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a5686adb-f120-4f74-9e47-4709cb6a6839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:09.915544 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:09.977788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1241/2471] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:36:09.987806 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:09.989609 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:10.047453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5686adb-f120-4f74-9e47-4709cb6a6839 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:10.048941 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a5686adb-f120-4f74-9e47-4709cb6a6839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:10.062706 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5686adb-f120-4f74-9e47-4709cb6a6839 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:10.063316 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c9c27-c9bd-4956-a0fb-55078ee51bd4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:10.067217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599d18a-8c28-4c40-8503-b461b8453295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:10.067798 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:10.067946 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599d18a-8c28-4c40-8503-b461b8453295 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:10.068095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599d18a-8c28-4c40-8503-b461b8453295 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:10.070404 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599d18a-8c28-4c40-8503-b461b8453295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:10.072379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:10.072379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:10.072379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] PATH_INFO: `/v3/users/7e9b10419f834e0a80d1f0a6023702c2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:10.072379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e9b10419f834e0a80d1f0a6023702c2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:10.074236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:10.092042 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599d18a-8c28-4c40-8503-b461b8453295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:10.093479 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599d18a-8c28-4c40-8503-b461b8453295 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:10.103858 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1231/2472] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:10.132503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e9b10419f834e0a80d1f0a6023702c2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.133143 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e9b10419f834e0a80d1f0a6023702c2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.148525 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e9b10419f834e0a80d1f0a6023702c2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.156617 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e9b10419f834e0a80d1f0a6023702c2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.157548 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e9b10419f834e0a80d1f0a6023702c2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.175041 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1242/2473] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:36:10 2022] DELETE /identity/v3/users/7e9b10419f834e0a80d1f0a6023702c2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:10.179665 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42e4de-78cf-47f2-ae7c-761029138d8f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:10.179883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42e4de-78cf-47f2-ae7c-761029138d8f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:10.180148 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42e4de-78cf-47f2-ae7c-761029138d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:10.206607 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5f42e4de-78cf-47f2-ae7c-761029138d8f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:10.207892 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5f42e4de-78cf-47f2-ae7c-761029138d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:10.288121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1232/2474] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:10.299792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:10.301363 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:10.372524 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b66b3-cfd0-4062-92da-eb38842cea7f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:10.376309 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42d485-d8a5-4645-8849-824e38f8d10e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:10.376867 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42d485-d8a5-4645-8849-824e38f8d10e service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:10.377147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42d485-d8a5-4645-8849-824e38f8d10e service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:10.377450 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42d485-d8a5-4645-8849-824e38f8d10e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:10.388635 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42d485-d8a5-4645-8849-824e38f8d10e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:10.390834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42d485-d8a5-4645-8849-824e38f8d10e service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:10.401314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1243/2475] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:10.792194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42e4de-78cf-47f2-ae7c-761029138d8f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:10.794120 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5f42e4de-78cf-47f2-ae7c-761029138d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:10.811209 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42e4de-78cf-47f2-ae7c-761029138d8f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:10.816876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:10.817437 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:10.817736 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:10.818064 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] PATH_INFO: `/v3/projects/6cbfcec8fd0b42c2b66641b5238083e9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:10.819300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cbfcec8fd0b42c2b66641b5238083e9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:10.821633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:10.836294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:10.906564 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cbfcec8fd0b42c2b66641b5238083e9', 'request_id': 'req-638f8439-aee4-4979-9a04-4a75437ad9bf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.914207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cbfcec8fd0b42c2b66641b5238083e9', 'request_id': 'req-638f8439-aee4-4979-9a04-4a75437ad9bf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.923786 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cbfcec8fd0b42c2b66641b5238083e9', 'request_id': 'req-638f8439-aee4-4979-9a04-4a75437ad9bf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.937989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023cba-64dc-4f41-9607-b1ef9246c6d5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:10.938270 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023cba-64dc-4f41-9607-b1ef9246c6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:10.938578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023cba-64dc-4f41-9607-b1ef9246c6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:10.940935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] The token cache is being invalidate because project 6cbfcec8fd0b42c2b66641b5238083e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:36:10.941423 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.942150 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:10.943978 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1233/2476] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:36:10 2022] DELETE /identity/v3/projects/6cbfcec8fd0b42c2b66641b5238083e9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:10.972936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-73023cba-64dc-4f41-9607-b1ef9246c6d5 None None] MFA Rules not processed for user `90e4dd806c804309aece8b368e0fb67e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:10.974312 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-73023cba-64dc-4f41-9607-b1ef9246c6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:11.002998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:11.060528 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1244/2477] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:36:11.075629 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:11.078178 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:11.148040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f8439-aee4-4979-9a04-4a75437ad9bf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:11.151971 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:11.152602 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:11.152880 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:11.153151 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:11.162782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:11.164238 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:11.174719 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1234/2478] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:11.268215 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023cba-64dc-4f41-9607-b1ef9246c6d5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:11.269811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-73023cba-64dc-4f41-9607-b1ef9246c6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:11.282526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023cba-64dc-4f41-9607-b1ef9246c6d5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:11.286219 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25551a82-5175-4f65-9eab-59cc8e70753d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:11.286929 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25551a82-5175-4f65-9eab-59cc8e70753d service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:11.287267 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25551a82-5175-4f65-9eab-59cc8e70753d service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:11.287598 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25551a82-5175-4f65-9eab-59cc8e70753d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:11.290529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-25551a82-5175-4f65-9eab-59cc8e70753d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:11.380430 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25551a82-5175-4f65-9eab-59cc8e70753d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:11.382453 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25551a82-5175-4f65-9eab-59cc8e70753d service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:11.404752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1245/2479] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:11.670646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:11.672164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:11.728895 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255535e-ea1a-4af8-99b7-f898ce57024e service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:11.732625 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:11.733218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:11.733500 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:11.733769 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:11.735668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:11.794247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:11.799277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:11.801070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:11.811900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1235/2480] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:12.546826 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80347fbc-361b-44d1-8639-af3c4d9f5011 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:12.547342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80347fbc-361b-44d1-8639-af3c4d9f5011 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:12.547342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80347fbc-361b-44d1-8639-af3c4d9f5011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:12.566667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-80347fbc-361b-44d1-8639-af3c4d9f5011 None None] MFA Rules not processed for user `90e4dd806c804309aece8b368e0fb67e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:12.567859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-80347fbc-361b-44d1-8639-af3c4d9f5011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:12.575424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1246/2481] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:12.929963 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:12.942159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8324-345a-4f03-9f43-9edccf69e3ef service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:12.948551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:12.950840 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:12.953633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:12.953925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:12.955980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:12.970255 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:12.972452 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:12.984687 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1236/2482] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:14.524022 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80347fbc-361b-44d1-8639-af3c4d9f5011 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:14.525522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-80347fbc-361b-44d1-8639-af3c4d9f5011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:14.604592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80347fbc-361b-44d1-8639-af3c4d9f5011 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:14.610088 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:14.610795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:14.611080 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:14.611982 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:14.613998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:14.633186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:14.636388 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:14.647958 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1247/2483] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:18.994589 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:18.996040 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:19.014579 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef79e38-b63f-4459-a19a-db32b58fc150 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:19.025578 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:19.026721 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:19.027073 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:19.027855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] PATH_INFO: `/v3/users/a5b7190649694b9882ab34c0f3114a3e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:19.030429 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5b7190649694b9882ab34c0f3114a3e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:19.032509 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:19.091817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5b7190649694b9882ab34c0f3114a3e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.095124 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5b7190649694b9882ab34c0f3114a3e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.103633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5b7190649694b9882ab34c0f3114a3e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.124037 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5b7190649694b9882ab34c0f3114a3e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.124689 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5b7190649694b9882ab34c0f3114a3e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.137236 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1237/2484] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:36:18 2022] DELETE /identity/v3/users/a5b7190649694b9882ab34c0f3114a3e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:19.395899 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:19.408006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d52a22-0f97-4617-9fe6-4c220a0a5696 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:19.422170 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:19.471967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:19.472725 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:19.473104 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:19.473503 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] PATH_INFO: `/v3/projects/0e87741d68c74ae29ecea930893e3449` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:19.474755 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e87741d68c74ae29ecea930893e3449)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:19.477015 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:19.525327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e87741d68c74ae29ecea930893e3449', 'request_id': 'req-afc11abc-362d-42c6-b4ce-688b4e8ec134'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.531864 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e87741d68c74ae29ecea930893e3449', 'request_id': 'req-afc11abc-362d-42c6-b4ce-688b4e8ec134'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.540785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e87741d68c74ae29ecea930893e3449', 'request_id': 'req-afc11abc-362d-42c6-b4ce-688b4e8ec134'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.551172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] The token cache is being invalidate because project 0e87741d68c74ae29ecea930893e3449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:36:19.551460 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.552633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.554141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1248/2485] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:36:19 2022] DELETE /identity/v3/projects/0e87741d68c74ae29ecea930893e3449 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:19.558211 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:19.559754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:19.574062 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35560af1-4b20-4c80-a699-e31be191dbe6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:19.631972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:19.632605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:19.632885 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:19.633154 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] PATH_INFO: `/v3/users/839c4fdf3b1a4506b7d0ca13629e9242` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:19.648591 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=839c4fdf3b1a4506b7d0ca13629e9242)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:19.651640 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:19.698949 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:19.702061 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839c4fdf3b1a4506b7d0ca13629e9242'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.703380 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839c4fdf3b1a4506b7d0ca13629e9242'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.717304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839c4fdf3b1a4506b7d0ca13629e9242'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.730016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839c4fdf3b1a4506b7d0ca13629e9242'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.731020 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839c4fdf3b1a4506b7d0ca13629e9242'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:19.746606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1238/2486] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:36:19 2022] DELETE /identity/v3/users/839c4fdf3b1a4506b7d0ca13629e9242 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:19.923639 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:19.925112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:19.949578 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc11abc-362d-42c6-b4ce-688b4e8ec134 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:20.007504 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:20.008913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:20.009505 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:20.009953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] PATH_INFO: `/v3/projects/4d06020a20de48389f76209233fea396` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:20.020700 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d06020a20de48389f76209233fea396)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:20.022653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:20.063110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:20.079343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d06020a20de48389f76209233fea396', 'request_id': 'req-5506b024-e174-44f2-a266-caae22feb1b8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:20.086361 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d06020a20de48389f76209233fea396', 'request_id': 'req-5506b024-e174-44f2-a266-caae22feb1b8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:20.095522 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d06020a20de48389f76209233fea396', 'request_id': 'req-5506b024-e174-44f2-a266-caae22feb1b8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:20.108417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] The token cache is being invalidate because project 4d06020a20de48389f76209233fea396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:36:20.108938 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:20.109694 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5506b024-e174-44f2-a266-caae22feb1b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:36:20.111325 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1249/2487] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:36:19 2022] DELETE /identity/v3/projects/4d06020a20de48389f76209233fea396 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:36:27.349016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:27.351263 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:27.440477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd312a-649d-41e8-add9-df76a5eaaa5f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:27.505937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e53c9-e73b-40ae-b478-a74dc99d7a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:27.506563 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e53c9-e73b-40ae-b478-a74dc99d7a7a service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:27.506834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e53c9-e73b-40ae-b478-a74dc99d7a7a service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:27.507096 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e53c9-e73b-40ae-b478-a74dc99d7a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:27.508802 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f8e53c9-e73b-40ae-b478-a74dc99d7a7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:27.520310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e53c9-e73b-40ae-b478-a74dc99d7a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:27.521861 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e53c9-e73b-40ae-b478-a74dc99d7a7a service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:27.532742 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1239/2488] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:36:30.104918 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:33.934095 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1dea2-0054-4aa0-8d1d-7eaf39c6d32f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:33.934785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1dea2-0054-4aa0-8d1d-7eaf39c6d32f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:33.934785 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1dea2-0054-4aa0-8d1d-7eaf39c6d32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:33.957940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-aae1dea2-0054-4aa0-8d1d-7eaf39c6d32f None None] MFA Rules not processed for user `90e4dd806c804309aece8b368e0fb67e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:33.959384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aae1dea2-0054-4aa0-8d1d-7eaf39c6d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:34.019264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1250/2489] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:37.528331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=4,keystone:SELECT=75,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:44.012438 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=25 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:36:48.829520 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3a043-b972-48fc-be05-c901f43d2b25 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:48.830137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3a043-b972-48fc-be05-c901f43d2b25 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:48.830137 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3a043-b972-48fc-be05-c901f43d2b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:48.850703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8bb3a043-b972-48fc-be05-c901f43d2b25 None None] MFA Rules not processed for user `e5efdc755c6a43f9b8cf570ac38b25eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:48.851923 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8bb3a043-b972-48fc-be05-c901f43d2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:48.906917 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1240/2490] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:56.507919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d227-5f99-474f-9555-336e9554fe65 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:56.509007 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d227-5f99-474f-9555-336e9554fe65 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:56.509112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d227-5f99-474f-9555-336e9554fe65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:56.529211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6d95d227-5f99-474f-9555-336e9554fe65 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:36:56.530550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6d95d227-5f99-474f-9555-336e9554fe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:56.586954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1251/2491] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:36:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:36:56.633695 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3a043-b972-48fc-be05-c901f43d2b25 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:36:56.635305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8bb3a043-b972-48fc-be05-c901f43d2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:36:56.702993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3a043-b972-48fc-be05-c901f43d2b25 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:36:56.707956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd596158-56e4-4c2e-975e-a32987d505f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:36:56.708554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd596158-56e4-4c2e-975e-a32987d505f1 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:36:56.708827 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd596158-56e4-4c2e-975e-a32987d505f1 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:36:56.709094 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd596158-56e4-4c2e-975e-a32987d505f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:36:56.717300 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd596158-56e4-4c2e-975e-a32987d505f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:36:56.718837 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd596158-56e4-4c2e-975e-a32987d505f1 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:36:56.726817 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1241/2492] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:36:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:37:06.580272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=25 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:37:06.723604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=57 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:37:11.945633 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06cb4-da68-42c5-85a0-4deb0e9a405b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:37:11.946127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06cb4-da68-42c5-85a0-4deb0e9a405b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:37:11.946304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d06cb4-da68-42c5-85a0-4deb0e9a405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:37:11.965110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-07d06cb4-da68-42c5-85a0-4deb0e9a405b None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:37:11.966286 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-07d06cb4-da68-42c5-85a0-4deb0e9a405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:37:11.973244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1252/2493] 10.222.0.16 () {58 vars in 967 bytes} [Mon Oct 17 10:37:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:37:11.986541 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd596158-56e4-4c2e-975e-a32987d505f1 service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:37:11.998013 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd596158-56e4-4c2e-975e-a32987d505f1 service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:37:12.002951 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:37:12.003537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:37:12.003812 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:37:12.004083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:37:12.012285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:37:12.013875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:37:12.021898 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1242/2494] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:37:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 10:37:13.860979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d06cb4-da68-42c5-85a0-4deb0e9a405b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:37:13.862386 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-07d06cb4-da68-42c5-85a0-4deb0e9a405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:37:13.922463 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:37:13.924533 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:37:13.939352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d06cb4-da68-42c5-85a0-4deb0e9a405b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:37:13.944496 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:37:13.945102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:37:13.946004 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:37:13.946269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:37:13.948023 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:37:13.959773 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:37:13.961717 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:37:13.971931 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1253/2495] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:37:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:37:13.996291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd8abd-4f1b-48e6-bc47-5023ab7cee5c service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:37:14.000954 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:37:14.001658 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:37:14.001995 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:37:14.002406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] PATH_INFO: `/v3/users/90e4dd806c804309aece8b368e0fb67e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:37:14.003712 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e4dd806c804309aece8b368e0fb67e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:37:14.005663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:37:14.052921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e4dd806c804309aece8b368e0fb67e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.053551 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e4dd806c804309aece8b368e0fb67e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.062632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e4dd806c804309aece8b368e0fb67e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.075883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e4dd806c804309aece8b368e0fb67e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.076555 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e4dd806c804309aece8b368e0fb67e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.089122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1243/2496] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:37:13 2022] DELETE /identity/v3/users/90e4dd806c804309aece8b368e0fb67e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:37:14.259163 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:37:14.270535 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a926782-48d7-4f95-af34-a5e14e0a5241 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:37:14.331097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:37:14.331684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:37:14.331965 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:37:14.332235 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] PATH_INFO: `/v3/projects/711ab39e17604b0d9df8ef228108396c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:37:14.333352 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711ab39e17604b0d9df8ef228108396c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:37:14.335295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:37:14.382684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711ab39e17604b0d9df8ef228108396c', 'request_id': 'req-a50c9882-6d84-4640-ae2b-76ec081c4813'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.388787 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711ab39e17604b0d9df8ef228108396c', 'request_id': 'req-a50c9882-6d84-4640-ae2b-76ec081c4813'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.395675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711ab39e17604b0d9df8ef228108396c', 'request_id': 'req-a50c9882-6d84-4640-ae2b-76ec081c4813'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.405740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] The token cache is being invalidate because project 711ab39e17604b0d9df8ef228108396c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:37:14.406025 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.406707 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a50c9882-6d84-4640-ae2b-76ec081c4813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:37:14.408248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1254/2497] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:37:14 2022] DELETE /identity/v3/projects/711ab39e17604b0d9df8ef228108396c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:37:24.085176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:37:24.402859 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:38:10.626457 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:38:10.627964 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:38:10.702468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc3004-f0ec-46d9-8bf2-1f17b5d84016 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:38:10.766436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:38:10.767044 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:38:10.767317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:38:10.767585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:38:10.769313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:38:10.826317 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:38:10.827915 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:38:10.836770 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1244/2498] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:38:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:38:20.832674 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:38:30.253264 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f19927-2541-4c4a-a74e-b48d7d0a8dcd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:38:30.253756 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f19927-2541-4c4a-a74e-b48d7d0a8dcd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:38:30.253862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f19927-2541-4c4a-a74e-b48d7d0a8dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:38:30.274200 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-41f19927-2541-4c4a-a74e-b48d7d0a8dcd None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:38:30.275516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-41f19927-2541-4c4a-a74e-b48d7d0a8dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:38:30.332209 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1255/2499] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:38:30.380202 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:38:30.381586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:38:30.449985 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa556b3-6501-404d-bf02-fa610321d703 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:38:30.455111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076e0bf-0f63-47f6-9271-000bb645a0b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:38:30.455750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076e0bf-0f63-47f6-9271-000bb645a0b7 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:38:30.456807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076e0bf-0f63-47f6-9271-000bb645a0b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:38:30.456807 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076e0bf-0f63-47f6-9271-000bb645a0b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:38:30.464620 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076e0bf-0f63-47f6-9271-000bb645a0b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:38:30.466944 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076e0bf-0f63-47f6-9271-000bb645a0b7 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:38:30.474916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1245/2500] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:38:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:38:40.324877 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=25 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:38:40.471774 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=32 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:38:45.950675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9105-8620-4cdb-abbd-e3219a0821a2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:38:45.951180 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9105-8620-4cdb-abbd-e3219a0821a2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:38:45.951305 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9105-8620-4cdb-abbd-e3219a0821a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:38:45.970641 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bbbc9105-8620-4cdb-abbd-e3219a0821a2 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:38:45.971816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bbbc9105-8620-4cdb-abbd-e3219a0821a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:38:45.978870 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1256/2501] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Oct 17 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:38:45.991932 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076e0bf-0f63-47f6-9271-000bb645a0b7 service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:38:46.002318 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076e0bf-0f63-47f6-9271-000bb645a0b7 service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:38:46.007295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad6541-4524-48be-b1b8-4022fe785aa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:38:46.007876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6541-4524-48be-b1b8-4022fe785aa5 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:38:46.008156 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6541-4524-48be-b1b8-4022fe785aa5 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:38:46.008428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6541-4524-48be-b1b8-4022fe785aa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:38:46.016468 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad6541-4524-48be-b1b8-4022fe785aa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:38:46.018057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad6541-4524-48be-b1b8-4022fe785aa5 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:38:46.025901 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1246/2502] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:38:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:38:55.965201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:38:56.022718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:39:00.318274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9105-8620-4cdb-abbd-e3219a0821a2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:39:00.319679 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bbbc9105-8620-4cdb-abbd-e3219a0821a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:39:00.391795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9105-8620-4cdb-abbd-e3219a0821a2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:39:00.396930 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:39:00.397554 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:39:00.397834 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:39:00.398107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:39:00.399865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:39:00.411954 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:39:00.413532 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:39:00.424948 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1257/2503] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:39:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:39:04.604560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7060-cf87-4567-85f5-8d5bf5d1b8f1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:39:04.605051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7060-cf87-4567-85f5-8d5bf5d1b8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:39:04.605157 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7060-cf87-4567-85f5-8d5bf5d1b8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:39:04.626241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-491c7060-cf87-4567-85f5-8d5bf5d1b8f1 None None] MFA Rules not processed for user `8b862b2975a14e01a5aec3aa936e9375`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:39:04.627478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-491c7060-cf87-4567-85f5-8d5bf5d1b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:39:04.634475 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1247/2504] 10.222.0.16 () {58 vars in 967 bytes} [Mon Oct 17 10:39:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:39:04.646730 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:39:04.658116 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef6e21-eb7e-4f9e-9e1d-58d9d30702f6 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:39:04.662959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf85319814ff4478b40b8a2e28e054ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:39:04.663545 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:39:04.663819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:39:04.664085 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:39:04.673072 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:39:04.674708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:39:04.682777 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1258/2505] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Oct 17 10:39:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 10:39:14.619585 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:39:14.679479 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=38 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:39:53.926836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c7060-cf87-4567-85f5-8d5bf5d1b8f1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:39:53.928295 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-491c7060-cf87-4567-85f5-8d5bf5d1b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:39:53.984240 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c7060-cf87-4567-85f5-8d5bf5d1b8f1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:39:53.987820 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:39:53.988410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:39:53.988685 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:39:53.988958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:39:53.990675 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:39:54.003748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:39:54.005141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:39:54.015969 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1248/2506] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:39:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:39:55.206534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:39:55.207946 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:39:55.219789 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecbcec-46f4-4dd3-a5ef-fd51b87987b2 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:39:55.223375 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:39:55.223967 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:39:55.224245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:39:55.224542 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:39:55.226236 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:39:55.239573 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:39:55.240988 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:39:55.253855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1259/2507] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:39:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:01.838857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:01.848464 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d237774-763c-4b34-a027-05a66f3ab420 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:01.852065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:01.852646 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:01.852920 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:01.853186 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:01.854900 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:01.916773 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:01.918176 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:01.927865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1249/2508] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:05.047425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.049541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:05.066156 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf2867-fde9-47ff-ab56-3054bd0ae39a service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.076289 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.077295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.078644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.078977 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.081176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:05.095364 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.097813 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:05.119281 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1260/2509] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:05.119556 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.120010 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:05.134017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336891e-1550-4729-b36e-466f2f5ce0f4 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.138857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.139481 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.139808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.140082 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] PATH_INFO: `/v3/users/e5efdc755c6a43f9b8cf570ac38b25eb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.141521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5efdc755c6a43f9b8cf570ac38b25eb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.144681 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:05.192517 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5efdc755c6a43f9b8cf570ac38b25eb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.193171 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5efdc755c6a43f9b8cf570ac38b25eb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.200531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5efdc755c6a43f9b8cf570ac38b25eb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.212882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5efdc755c6a43f9b8cf570ac38b25eb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.213512 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9e90e433-f4f2-48fa-b243-007bcd6522a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5efdc755c6a43f9b8cf570ac38b25eb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.226207 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1250/2510] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:40:05 2022] DELETE /identity/v3/users/e5efdc755c6a43f9b8cf570ac38b25eb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:40:05.423840 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.441350 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ae028-17ac-418e-a8f2-6bf103fcf4dc service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.500064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.500692 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.500972 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.501269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] PATH_INFO: `/v3/projects/6a41305c4efa44edb98d1ca076ccbdff` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.502487 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a41305c4efa44edb98d1ca076ccbdff)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.504398 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:05.551003 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a41305c4efa44edb98d1ca076ccbdff', 'request_id': 'req-979e8226-bf27-4e2e-bec2-864b3f20c4d4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.557021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a41305c4efa44edb98d1ca076ccbdff', 'request_id': 'req-979e8226-bf27-4e2e-bec2-864b3f20c4d4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.564761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a41305c4efa44edb98d1ca076ccbdff', 'request_id': 'req-979e8226-bf27-4e2e-bec2-864b3f20c4d4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.575357 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] The token cache is being invalidate because project 6a41305c4efa44edb98d1ca076ccbdff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:40:05.575643 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.576271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-979e8226-bf27-4e2e-bec2-864b3f20c4d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:05.577748 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1261/2511] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:40:05 2022] DELETE /identity/v3/projects/6a41305c4efa44edb98d1ca076ccbdff => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:05.583498 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2ced5-eac5-4a43-a13f-1fc7b7e565b3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.583781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2ced5-eac5-4a43-a13f-1fc7b7e565b3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.584048 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2ced5-eac5-4a43-a13f-1fc7b7e565b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.603835 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eed2ced5-eac5-4a43-a13f-1fc7b7e565b3 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:05.605017 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eed2ced5-eac5-4a43-a13f-1fc7b7e565b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:05.609366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:05.665291 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1251/2512] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:05.673512 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e877bd-15ec-4873-86b9-e678a67391b5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.673793 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e877bd-15ec-4873-86b9-e678a67391b5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.674066 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e877bd-15ec-4873-86b9-e678a67391b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.694036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b8e877bd-15ec-4873-86b9-e678a67391b5 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:05.695247 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b8e877bd-15ec-4873-86b9-e678a67391b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:05.702920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1262/2513] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:05.707716 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2ced5-eac5-4a43-a13f-1fc7b7e565b3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.717090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2ced5-eac5-4a43-a13f-1fc7b7e565b3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.721694 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.722252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.722554 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.722821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.723458 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:40:05.723754 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.725326 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:05.736882 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1252/2514] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:40:05.741956 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e877bd-15ec-4873-86b9-e678a67391b5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.751768 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e877bd-15ec-4873-86b9-e678a67391b5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.756447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5cd25-3536-4a7d-836b-55e594dd0aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.757045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5cd25-3536-4a7d-836b-55e594dd0aca admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.757324 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5cd25-3536-4a7d-836b-55e594dd0aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.757591 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5cd25-3536-4a7d-836b-55e594dd0aca admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.758224 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5cd25-3536-4a7d-836b-55e594dd0aca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.760174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5cd25-3536-4a7d-836b-55e594dd0aca admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:05.790855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1263/2515] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:40:05 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:40:05.792104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.803810 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8467a8-1d94-4ae4-b53c-ef7d225ced3f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.862217 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba370f-c2ae-43c0-9055-eb47ba282b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.862841 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba370f-c2ae-43c0-9055-eb47ba282b10 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.863117 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba370f-c2ae-43c0-9055-eb47ba282b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.863387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba370f-c2ae-43c0-9055-eb47ba282b10 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.864016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba370f-c2ae-43c0-9055-eb47ba282b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.865906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba370f-c2ae-43c0-9055-eb47ba282b10 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:05.887015 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1253/2516] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:40:05 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:40:05.891492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5cd25-3536-4a7d-836b-55e594dd0aca admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.902257 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5cd25-3536-4a7d-836b-55e594dd0aca admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.906866 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602138a-4445-423e-8e5e-f57d912ed60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.907562 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602138a-4445-423e-8e5e-f57d912ed60a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.907862 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602138a-4445-423e-8e5e-f57d912ed60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.908110 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602138a-4445-423e-8e5e-f57d912ed60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.908750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602138a-4445-423e-8e5e-f57d912ed60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.910355 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602138a-4445-423e-8e5e-f57d912ed60a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:05.920340 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1264/2517] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:05.924560 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba370f-c2ae-43c0-9055-eb47ba282b10 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.934373 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba370f-c2ae-43c0-9055-eb47ba282b10 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.939586 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e12576-fb65-4466-9742-f438a8c139a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.940164 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12576-fb65-4466-9742-f438a8c139a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.940436 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12576-fb65-4466-9742-f438a8c139a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.940703 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12576-fb65-4466-9742-f438a8c139a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.941332 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e12576-fb65-4466-9742-f438a8c139a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.942929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e12576-fb65-4466-9742-f438a8c139a2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:05.953374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1254/2518] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:05.958101 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602138a-4445-423e-8e5e-f57d912ed60a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:05.967951 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602138a-4445-423e-8e5e-f57d912ed60a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:05.972701 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:05.973740 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:05.974188 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:05.974664 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] PATH_INFO: `/v3/projects/1b580463e35646b583faee9675f11e60/users/273b683e939347eabf1e4730481791f0/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:05.989943 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b580463e35646b583faee9675f11e60, user_id=273b683e939347eabf1e4730481791f0, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:05.993750 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:06.010884 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:06.016129 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1265/2519] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:05 2022] PUT /identity/v3/projects/1b580463e35646b583faee9675f11e60/users/273b683e939347eabf1e4730481791f0/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:06.026815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5e29-daf6-44f0-80ad-0ce71ff41b41 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:06.027098 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5e29-daf6-44f0-80ad-0ce71ff41b41 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:06.027366 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5e29-daf6-44f0-80ad-0ce71ff41b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:06.055413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a94f5e29-daf6-44f0-80ad-0ce71ff41b41 None None] MFA Rules not processed for user `273b683e939347eabf1e4730481791f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:06.056606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a94f5e29-daf6-44f0-80ad-0ce71ff41b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:06.120984 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1255/2520] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:06.133074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:06.134861 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:06.208262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9903ad4-756e-4882-aa5f-a0c7b21583c8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:06.213658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9016b-2ce7-4436-9df7-27dcce4d4c85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:06.214256 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9016b-2ce7-4436-9df7-27dcce4d4c85 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:06.214565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9016b-2ce7-4436-9df7-27dcce4d4c85 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:06.214838 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9016b-2ce7-4436-9df7-27dcce4d4c85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:06.222315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9016b-2ce7-4436-9df7-27dcce4d4c85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:06.223904 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9016b-2ce7-4436-9df7-27dcce4d4c85 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:06.233097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1266/2521] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:14.589267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f5e29-daf6-44f0-80ad-0ce71ff41b41 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:14.591597 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:14.649403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f5e29-daf6-44f0-80ad-0ce71ff41b41 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:14.654016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:14.654653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:14.654927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:14.655191 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] PATH_INFO: `/v3/users/273b683e939347eabf1e4730481791f0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:14.656285 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=273b683e939347eabf1e4730481791f0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:14.658956 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:14.704080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273b683e939347eabf1e4730481791f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:14.704732 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273b683e939347eabf1e4730481791f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:14.711845 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273b683e939347eabf1e4730481791f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:14.724172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273b683e939347eabf1e4730481791f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:14.724737 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273b683e939347eabf1e4730481791f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:14.738014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1256/2522] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:40:14 2022] DELETE /identity/v3/users/273b683e939347eabf1e4730481791f0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:14.742712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b185a07-ef05-46ab-8e62-66c1c16534c8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:14.743191 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b185a07-ef05-46ab-8e62-66c1c16534c8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:14.743295 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b185a07-ef05-46ab-8e62-66c1c16534c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:14.762612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0b185a07-ef05-46ab-8e62-66c1c16534c8 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:14.763779 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b185a07-ef05-46ab-8e62-66c1c16534c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:14.820922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1267/2523] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:40:14.831435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:14.832853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:14.891341 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bba30-b4cd-4af5-9f7d-06204ffe9c47 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:14.895515 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7f6ee-924d-4e7d-932c-183fcb2196cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:14.896109 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7f6ee-924d-4e7d-932c-183fcb2196cb service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:14.896382 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7f6ee-924d-4e7d-932c-183fcb2196cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:14.896648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7f6ee-924d-4e7d-932c-183fcb2196cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:14.905489 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7f6ee-924d-4e7d-932c-183fcb2196cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:14.906943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7f6ee-924d-4e7d-932c-183fcb2196cb service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:14.916113 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1257/2524] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:15.185426 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b185a07-ef05-46ab-8e62-66c1c16534c8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.186854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b185a07-ef05-46ab-8e62-66c1c16534c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:15.201434 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b185a07-ef05-46ab-8e62-66c1c16534c8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.207439 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.208250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.208529 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.208795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] PATH_INFO: `/v3/projects/1b580463e35646b583faee9675f11e60` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.209919 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b580463e35646b583faee9675f11e60)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.211957 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:15.259769 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b580463e35646b583faee9675f11e60', 'request_id': 'req-1ffc81ff-dd84-480b-9532-122a7548e57d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:15.265874 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b580463e35646b583faee9675f11e60', 'request_id': 'req-1ffc81ff-dd84-480b-9532-122a7548e57d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:15.273549 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b580463e35646b583faee9675f11e60', 'request_id': 'req-1ffc81ff-dd84-480b-9532-122a7548e57d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:15.284429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] The token cache is being invalidate because project 1b580463e35646b583faee9675f11e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:40:15.284708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:15.285314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1ffc81ff-dd84-480b-9532-122a7548e57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:15.287213 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1268/2525] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:40:15 2022] DELETE /identity/v3/projects/1b580463e35646b583faee9675f11e60 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:40:15.291864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8443117-bf25-43ac-9fc1-af281ef71797 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.292162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8443117-bf25-43ac-9fc1-af281ef71797 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.292430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8443117-bf25-43ac-9fc1-af281ef71797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.311881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c8443117-bf25-43ac-9fc1-af281ef71797 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:15.313049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c8443117-bf25-43ac-9fc1-af281ef71797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:15.333559 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:15.375538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1258/2526] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 10:40:15.384986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d2cb-5569-4acb-be5c-de260e9fba17 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.385266 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d2cb-5569-4acb-be5c-de260e9fba17 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.385537 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d2cb-5569-4acb-be5c-de260e9fba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.405975 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fda4d2cb-5569-4acb-be5c-de260e9fba17 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:15.407745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fda4d2cb-5569-4acb-be5c-de260e9fba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:15.415461 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1269/2527] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:15.419735 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8443117-bf25-43ac-9fc1-af281ef71797 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.429750 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8443117-bf25-43ac-9fc1-af281ef71797 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.434162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.434781 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.435056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.435327 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.435961 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:40:15.436258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.437847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:15.449196 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1259/2528] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 10:40:15.453384 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4d2cb-5569-4acb-be5c-de260e9fba17 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.458274 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:15.464313 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4d2cb-5569-4acb-be5c-de260e9fba17 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.470654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0fd2f-1521-41ed-8635-c2776987f9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.471362 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0fd2f-1521-41ed-8635-c2776987f9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.471640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0fd2f-1521-41ed-8635-c2776987f9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.471907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0fd2f-1521-41ed-8635-c2776987f9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.472541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0fd2f-1521-41ed-8635-c2776987f9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.474458 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0fd2f-1521-41ed-8635-c2776987f9d1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:15.501615 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1270/2529] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:40:15 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:15.505884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.515855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971f877-11d2-44c7-a3ff-b02de3dd6cae admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.573050 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ef997-096d-4295-ac8f-92232ff85253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.573676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ef997-096d-4295-ac8f-92232ff85253 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.573950 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ef997-096d-4295-ac8f-92232ff85253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.574218 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ef997-096d-4295-ac8f-92232ff85253 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.574912 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ef997-096d-4295-ac8f-92232ff85253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.576875 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ef997-096d-4295-ac8f-92232ff85253 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:15.598267 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1260/2530] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:40:15 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:15.602371 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0fd2f-1521-41ed-8635-c2776987f9d1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.613385 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0fd2f-1521-41ed-8635-c2776987f9d1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.617817 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3802925-dd49-4c54-8bc3-07cdb605795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.618457 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3802925-dd49-4c54-8bc3-07cdb605795e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.618731 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3802925-dd49-4c54-8bc3-07cdb605795e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.618998 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3802925-dd49-4c54-8bc3-07cdb605795e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.619628 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3802925-dd49-4c54-8bc3-07cdb605795e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.621198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3802925-dd49-4c54-8bc3-07cdb605795e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:15.631516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1271/2531] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:15.635282 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ef997-096d-4295-ac8f-92232ff85253 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.645883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ef997-096d-4295-ac8f-92232ff85253 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.650532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0bf3f-5301-44fb-8d82-8218d4320685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.651118 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0bf3f-5301-44fb-8d82-8218d4320685 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.651395 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0bf3f-5301-44fb-8d82-8218d4320685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.651668 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0bf3f-5301-44fb-8d82-8218d4320685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.652308 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0bf3f-5301-44fb-8d82-8218d4320685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.653884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0bf3f-5301-44fb-8d82-8218d4320685 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:15.664868 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1261/2532] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:15.668816 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3802925-dd49-4c54-8bc3-07cdb605795e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.678619 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3802925-dd49-4c54-8bc3-07cdb605795e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.683088 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c3dad-044b-4065-b89e-9adb8b2608d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.683720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c3dad-044b-4065-b89e-9adb8b2608d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.683993 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c3dad-044b-4065-b89e-9adb8b2608d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.684261 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c3dad-044b-4065-b89e-9adb8b2608d2 admin admin] PATH_INFO: `/v3/projects/dfa68546597d414795c15ec0282d4ca8/users/f0337394935147bcad2cb0c700176ead/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.699034 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c3dad-044b-4065-b89e-9adb8b2608d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa68546597d414795c15ec0282d4ca8, user_id=f0337394935147bcad2cb0c700176ead, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.701667 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c3dad-044b-4065-b89e-9adb8b2608d2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:15.714567 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1272/2533] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:15 2022] PUT /identity/v3/projects/dfa68546597d414795c15ec0282d4ca8/users/f0337394935147bcad2cb0c700176ead/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:15.725090 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb2125-c85f-4567-9637-4ee576ed0067 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.725375 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb2125-c85f-4567-9637-4ee576ed0067 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.725648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb2125-c85f-4567-9637-4ee576ed0067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.754856 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-80fb2125-c85f-4567-9637-4ee576ed0067 None None] MFA Rules not processed for user `f0337394935147bcad2cb0c700176ead`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:15.756046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-80fb2125-c85f-4567-9637-4ee576ed0067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:15.818808 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1262/2534] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:15.823300 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c3dad-044b-4065-b89e-9adb8b2608d2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.884001 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c3dad-044b-4065-b89e-9adb8b2608d2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.888559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945510d1-7b1c-4a29-aa7c-042148e9747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.889154 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945510d1-7b1c-4a29-aa7c-042148e9747e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.889431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945510d1-7b1c-4a29-aa7c-042148e9747e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.889702 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945510d1-7b1c-4a29-aa7c-042148e9747e admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.890344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945510d1-7b1c-4a29-aa7c-042148e9747e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.892501 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945510d1-7b1c-4a29-aa7c-042148e9747e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:15.918651 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1273/2535] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:40:15 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:15.922728 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb2125-c85f-4567-9637-4ee576ed0067 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:15.933818 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb2125-c85f-4567-9637-4ee576ed0067 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:15.952047 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:15.990129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d07f1-f51c-49bb-840b-e5c81aa259fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:15.990826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d07f1-f51c-49bb-840b-e5c81aa259fb admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:15.991102 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d07f1-f51c-49bb-840b-e5c81aa259fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:15.991372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d07f1-f51c-49bb-840b-e5c81aa259fb admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:15.992004 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d07f1-f51c-49bb-840b-e5c81aa259fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:15.993897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d07f1-f51c-49bb-840b-e5c81aa259fb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:16.017460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1263/2536] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:40:15 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:40:16.019301 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945510d1-7b1c-4a29-aa7c-042148e9747e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:16.029409 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945510d1-7b1c-4a29-aa7c-042148e9747e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:16.034566 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508601df-2754-4d57-b47f-119923726efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:16.035164 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508601df-2754-4d57-b47f-119923726efa admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:16.035441 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508601df-2754-4d57-b47f-119923726efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:16.035711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508601df-2754-4d57-b47f-119923726efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:16.036343 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508601df-2754-4d57-b47f-119923726efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:16.037995 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508601df-2754-4d57-b47f-119923726efa admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:16.049102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1274/2537] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:16.053167 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d07f1-f51c-49bb-840b-e5c81aa259fb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:16.062996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d07f1-f51c-49bb-840b-e5c81aa259fb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:16.068185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b48db5-4e53-4400-8d86-db6a4496089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:16.068884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48db5-4e53-4400-8d86-db6a4496089e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:16.069106 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48db5-4e53-4400-8d86-db6a4496089e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:16.069377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48db5-4e53-4400-8d86-db6a4496089e admin admin] PATH_INFO: `/v3/projects/8ffeb7a9de65489e8d46b7e9760f091a/users/7e6d7e29cf78413f8773960b6c57b899/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:16.084931 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b48db5-4e53-4400-8d86-db6a4496089e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffeb7a9de65489e8d46b7e9760f091a, user_id=7e6d7e29cf78413f8773960b6c57b899, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:16.087631 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b48db5-4e53-4400-8d86-db6a4496089e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:16.100644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1264/2538] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:16 2022] PUT /identity/v3/projects/8ffeb7a9de65489e8d46b7e9760f091a/users/7e6d7e29cf78413f8773960b6c57b899/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:16.104608 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508601df-2754-4d57-b47f-119923726efa admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:16.115250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508601df-2754-4d57-b47f-119923726efa admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:16.171192 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063121e2-0526-46ae-9650-b8889f8549d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:16.171820 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063121e2-0526-46ae-9650-b8889f8549d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:16.172097 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063121e2-0526-46ae-9650-b8889f8549d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:16.172365 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063121e2-0526-46ae-9650-b8889f8549d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:16.173678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063121e2-0526-46ae-9650-b8889f8549d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:16.175333 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063121e2-0526-46ae-9650-b8889f8549d7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:16.184926 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1275/2539] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:16.188972 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b48db5-4e53-4400-8d86-db6a4496089e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:16.199027 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b48db5-4e53-4400-8d86-db6a4496089e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:16.203500 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:16.204172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:16.204444 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:16.204717 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] PATH_INFO: `/v3/projects/8ffeb7a9de65489e8d46b7e9760f091a/users/7e6d7e29cf78413f8773960b6c57b899/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:16.206632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffeb7a9de65489e8d46b7e9760f091a, user_id=7e6d7e29cf78413f8773960b6c57b899, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:16.209247 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:16.219129 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1265/2540] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:16 2022] PUT /identity/v3/projects/8ffeb7a9de65489e8d46b7e9760f091a/users/7e6d7e29cf78413f8773960b6c57b899/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:16.229749 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b306c-4fc6-4971-ae69-cd56135e9f35 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:16.230036 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b306c-4fc6-4971-ae69-cd56135e9f35 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:16.230312 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b306c-4fc6-4971-ae69-cd56135e9f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:16.258712 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-014b306c-4fc6-4971-ae69-cd56135e9f35 None None] MFA Rules not processed for user `7e6d7e29cf78413f8773960b6c57b899`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:16.259914 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-014b306c-4fc6-4971-ae69-cd56135e9f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:16.263783 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:16.328953 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1276/2541] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:16.340372 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:16.342078 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:16.399067 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d69d5-3c17-4ee0-89ce-b041330d0d16 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:16.402714 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4885973-302c-4b68-9e6b-d54776f5bf1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:16.403301 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4885973-302c-4b68-9e6b-d54776f5bf1e service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:16.403575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4885973-302c-4b68-9e6b-d54776f5bf1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:16.403847 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4885973-302c-4b68-9e6b-d54776f5bf1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:16.413403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4885973-302c-4b68-9e6b-d54776f5bf1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:16.414852 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4885973-302c-4b68-9e6b-d54776f5bf1e service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:16.424792 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1266/2542] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:16.809293 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b306c-4fc6-4971-ae69-cd56135e9f35 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:16.871271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b306c-4fc6-4971-ae69-cd56135e9f35 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:16.875670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:16.876782 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:16.877056 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:16.877327 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] PATH_INFO: `/v3/users/f0337394935147bcad2cb0c700176ead` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:16.878467 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0337394935147bcad2cb0c700176ead)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:16.880420 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:16.925565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0337394935147bcad2cb0c700176ead'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:16.926206 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0337394935147bcad2cb0c700176ead'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:16.938194 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0337394935147bcad2cb0c700176ead'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:16.945743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0337394935147bcad2cb0c700176ead'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:16.946366 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0337394935147bcad2cb0c700176ead'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:16.959107 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1277/2543] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:40:16 2022] DELETE /identity/v3/users/f0337394935147bcad2cb0c700176ead => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:16.963937 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f538d49-76b0-497a-bdac-68ebb79313be None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:16.964420 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f538d49-76b0-497a-bdac-68ebb79313be None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:16.964523 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f538d49-76b0-497a-bdac-68ebb79313be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:16.983142 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4f538d49-76b0-497a-bdac-68ebb79313be None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:16.984320 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f538d49-76b0-497a-bdac-68ebb79313be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:17.042990 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1267/2544] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:17.053147 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:17.054624 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:17.077021 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:17.114803 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714bac2-4662-4f35-93b2-6ef355a3dbb2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:17.118425 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b383b2d-12af-4652-97a8-f9979dd6473b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:17.119019 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b383b2d-12af-4652-97a8-f9979dd6473b service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:17.119296 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b383b2d-12af-4652-97a8-f9979dd6473b service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:17.119559 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b383b2d-12af-4652-97a8-f9979dd6473b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:17.128888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b383b2d-12af-4652-97a8-f9979dd6473b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:17.130246 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b383b2d-12af-4652-97a8-f9979dd6473b service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:17.139390 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1278/2545] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:17.405527 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f538d49-76b0-497a-bdac-68ebb79313be None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:17.407030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f538d49-76b0-497a-bdac-68ebb79313be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:17.420496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f538d49-76b0-497a-bdac-68ebb79313be None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:17.427415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:17.428056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:17.428315 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:17.428657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] PATH_INFO: `/v3/projects/dfa68546597d414795c15ec0282d4ca8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:17.429718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa68546597d414795c15ec0282d4ca8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:17.431745 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:17.445727 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:17.480571 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa68546597d414795c15ec0282d4ca8', 'request_id': 'req-07a78e07-971d-4828-b27c-2e5f766d24ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.487456 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa68546597d414795c15ec0282d4ca8', 'request_id': 'req-07a78e07-971d-4828-b27c-2e5f766d24ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.492927 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa68546597d414795c15ec0282d4ca8', 'request_id': 'req-07a78e07-971d-4828-b27c-2e5f766d24ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.506460 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] The token cache is being invalidate because project dfa68546597d414795c15ec0282d4ca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:40:17.506747 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.507361 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.508857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1268/2546] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:40:17 2022] DELETE /identity/v3/projects/dfa68546597d414795c15ec0282d4ca8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:17.512854 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b383b2d-12af-4652-97a8-f9979dd6473b service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:17.522673 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b383b2d-12af-4652-97a8-f9979dd6473b service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:17.578788 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:17.579865 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:17.580145 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:17.580413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] PATH_INFO: `/v3/users/7e6d7e29cf78413f8773960b6c57b899` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:17.581533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6d7e29cf78413f8773960b6c57b899)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:17.583486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:17.629413 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6d7e29cf78413f8773960b6c57b899'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.630051 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6d7e29cf78413f8773960b6c57b899'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.642055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6d7e29cf78413f8773960b6c57b899'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.649263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6d7e29cf78413f8773960b6c57b899'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.649823 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d62ae87-56c6-477f-9247-ecb9a2a78e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6d7e29cf78413f8773960b6c57b899'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:17.662781 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1279/2547] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:40:17 2022] DELETE /identity/v3/users/7e6d7e29cf78413f8773960b6c57b899 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:17.930184 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:17.931727 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:17.945994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a78e07-971d-4828-b27c-2e5f766d24ad admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.009855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.010558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.010826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.011093 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] PATH_INFO: `/v3/projects/8ffeb7a9de65489e8d46b7e9760f091a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.012241 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffeb7a9de65489e8d46b7e9760f091a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.014175 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.060455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffeb7a9de65489e8d46b7e9760f091a', 'request_id': 'req-0f2340ce-7562-4de3-96fa-20e75a882573'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:18.066839 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffeb7a9de65489e8d46b7e9760f091a', 'request_id': 'req-0f2340ce-7562-4de3-96fa-20e75a882573'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:18.072455 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffeb7a9de65489e8d46b7e9760f091a', 'request_id': 'req-0f2340ce-7562-4de3-96fa-20e75a882573'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:18.086605 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] The token cache is being invalidate because project 8ffeb7a9de65489e8d46b7e9760f091a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:40:18.086891 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:18.087504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f2340ce-7562-4de3-96fa-20e75a882573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:18.088981 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1269/2548] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:40:17 2022] DELETE /identity/v3/projects/8ffeb7a9de65489e8d46b7e9760f091a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:18.095498 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e1389-803d-43d6-a057-0fec94e941ee None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.096408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e1389-803d-43d6-a057-0fec94e941ee None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.096408 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e1389-803d-43d6-a057-0fec94e941ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.117763 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-793e1389-803d-43d6-a057-0fec94e941ee None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:18.119720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-793e1389-803d-43d6-a057-0fec94e941ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:18.160241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:18.183112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1280/2549] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:18.245698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd2865-09ef-4d95-a1d0-4a05babc7515 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.246116 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd2865-09ef-4d95-a1d0-4a05babc7515 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.246419 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd2865-09ef-4d95-a1d0-4a05babc7515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.267305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-05cd2865-09ef-4d95-a1d0-4a05babc7515 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:18.268823 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-05cd2865-09ef-4d95-a1d0-4a05babc7515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:18.276521 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1270/2550] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:18.280718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e1389-803d-43d6-a057-0fec94e941ee None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.291045 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e1389-803d-43d6-a057-0fec94e941ee None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.295481 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.296068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.296344 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.296612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.297252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:40:18.297550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.299175 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.310492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1281/2551] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:40:18.314718 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd2865-09ef-4d95-a1d0-4a05babc7515 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.323744 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd2865-09ef-4d95-a1d0-4a05babc7515 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.328406 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f717aba-8a7a-439a-8aa6-f8c59c467863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.329494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f717aba-8a7a-439a-8aa6-f8c59c467863 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.329776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f717aba-8a7a-439a-8aa6-f8c59c467863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.330026 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f717aba-8a7a-439a-8aa6-f8c59c467863 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.330698 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f717aba-8a7a-439a-8aa6-f8c59c467863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.332613 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f717aba-8a7a-439a-8aa6-f8c59c467863 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.359996 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1271/2552] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:40:18 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:40:18.364055 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.373855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c390087-8045-44f4-a9ad-1f80e94e56e9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.428855 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c848-69bf-4ab4-aec4-4055a403c27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.429451 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c848-69bf-4ab4-aec4-4055a403c27e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.429728 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c848-69bf-4ab4-aec4-4055a403c27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.429999 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c848-69bf-4ab4-aec4-4055a403c27e admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.430661 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91c848-69bf-4ab4-aec4-4055a403c27e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.432612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91c848-69bf-4ab4-aec4-4055a403c27e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.453648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1282/2553] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:40:18 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 10:40:18.458283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f717aba-8a7a-439a-8aa6-f8c59c467863 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.463653 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:18.470041 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f717aba-8a7a-439a-8aa6-f8c59c467863 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.475881 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4370b-19df-4153-98c7-f3c736476926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.476472 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4370b-19df-4153-98c7-f3c736476926 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.476748 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4370b-19df-4153-98c7-f3c736476926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.477018 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4370b-19df-4153-98c7-f3c736476926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.477660 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4370b-19df-4153-98c7-f3c736476926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.479271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4370b-19df-4153-98c7-f3c736476926 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.489473 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1272/2554] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:18.493462 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c848-69bf-4ab4-aec4-4055a403c27e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.504015 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c848-69bf-4ab4-aec4-4055a403c27e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.508658 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668d1dc-f4ab-43f4-914f-529f220b9d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.509244 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668d1dc-f4ab-43f4-914f-529f220b9d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.509523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668d1dc-f4ab-43f4-914f-529f220b9d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.509795 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668d1dc-f4ab-43f4-914f-529f220b9d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.510464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668d1dc-f4ab-43f4-914f-529f220b9d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.512042 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668d1dc-f4ab-43f4-914f-529f220b9d86 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.523250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1283/2555] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:18.527111 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4370b-19df-4153-98c7-f3c736476926 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.537526 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4370b-19df-4153-98c7-f3c736476926 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.542313 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33448ccb-db82-4ecc-b912-963ed571291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.543008 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33448ccb-db82-4ecc-b912-963ed571291e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.543277 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33448ccb-db82-4ecc-b912-963ed571291e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.543542 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33448ccb-db82-4ecc-b912-963ed571291e admin admin] PATH_INFO: `/v3/projects/63a39bd12d894b81b11978c31fd3d3a4/users/11400830569f4a5a8b2098e9195f6bde/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.558135 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33448ccb-db82-4ecc-b912-963ed571291e admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a39bd12d894b81b11978c31fd3d3a4, user_id=11400830569f4a5a8b2098e9195f6bde, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.560776 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33448ccb-db82-4ecc-b912-963ed571291e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.573813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1273/2556] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:18 2022] PUT /identity/v3/projects/63a39bd12d894b81b11978c31fd3d3a4/users/11400830569f4a5a8b2098e9195f6bde/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:40:18.585032 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0a96-65f3-4d5e-b4c5-7091c6131c1f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.585318 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0a96-65f3-4d5e-b4c5-7091c6131c1f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.585593 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0a96-65f3-4d5e-b4c5-7091c6131c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.613647 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ad9a0a96-65f3-4d5e-b4c5-7091c6131c1f None None] MFA Rules not processed for user `11400830569f4a5a8b2098e9195f6bde`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:18.614887 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ad9a0a96-65f3-4d5e-b4c5-7091c6131c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:18.678819 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1284/2557] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:18.683538 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33448ccb-db82-4ecc-b912-963ed571291e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.748866 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33448ccb-db82-4ecc-b912-963ed571291e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.753976 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da1d02-f5fb-485a-a406-a516a1611412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.754592 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da1d02-f5fb-485a-a406-a516a1611412 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.754865 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da1d02-f5fb-485a-a406-a516a1611412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.755128 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da1d02-f5fb-485a-a406-a516a1611412 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.755757 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da1d02-f5fb-485a-a406-a516a1611412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.757618 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da1d02-f5fb-485a-a406-a516a1611412 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.782830 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1274/2558] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:40:18 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:18.786897 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a0a96-65f3-4d5e-b4c5-7091c6131c1f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.797186 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a0a96-65f3-4d5e-b4c5-7091c6131c1f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.830431 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:18.852797 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa83f3-7c5b-4e8f-a47d-20cbc32cae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.853393 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa83f3-7c5b-4e8f-a47d-20cbc32cae79 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.853670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa83f3-7c5b-4e8f-a47d-20cbc32cae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.853938 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa83f3-7c5b-4e8f-a47d-20cbc32cae79 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.854595 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa83f3-7c5b-4e8f-a47d-20cbc32cae79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.856499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa83f3-7c5b-4e8f-a47d-20cbc32cae79 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.878205 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1285/2559] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:40:18 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:18.882594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da1d02-f5fb-485a-a406-a516a1611412 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.893343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da1d02-f5fb-485a-a406-a516a1611412 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.897883 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b730f32-435c-407c-9429-1e9822b09be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.898504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b730f32-435c-407c-9429-1e9822b09be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.898782 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b730f32-435c-407c-9429-1e9822b09be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.899052 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b730f32-435c-407c-9429-1e9822b09be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.899695 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b730f32-435c-407c-9429-1e9822b09be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.901259 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b730f32-435c-407c-9429-1e9822b09be6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.911635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1275/2560] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:18.915670 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa83f3-7c5b-4e8f-a47d-20cbc32cae79 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.926112 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa83f3-7c5b-4e8f-a47d-20cbc32cae79 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:18.930708 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebfc84-a9a3-4c0f-ad73-dd6408d5b6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:18.931349 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebfc84-a9a3-4c0f-ad73-dd6408d5b6b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:18.931627 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebfc84-a9a3-4c0f-ad73-dd6408d5b6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:18.931978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebfc84-a9a3-4c0f-ad73-dd6408d5b6b7 admin admin] PATH_INFO: `/v3/projects/71b5e8fd10ed4c2a8bdb1abce57809f8/users/fb4180717a2941f28323953296f3177c/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:18.946744 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebfc84-a9a3-4c0f-ad73-dd6408d5b6b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b5e8fd10ed4c2a8bdb1abce57809f8, user_id=fb4180717a2941f28323953296f3177c, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:18.949800 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebfc84-a9a3-4c0f-ad73-dd6408d5b6b7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:18.963336 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1286/2561] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:18 2022] PUT /identity/v3/projects/71b5e8fd10ed4c2a8bdb1abce57809f8/users/fb4180717a2941f28323953296f3177c/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:18.967262 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b730f32-435c-407c-9429-1e9822b09be6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:18.977918 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b730f32-435c-407c-9429-1e9822b09be6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:19.031980 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cedea-4500-4f8e-9999-f23ad2825f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:19.033386 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cedea-4500-4f8e-9999-f23ad2825f00 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:19.033513 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cedea-4500-4f8e-9999-f23ad2825f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:19.033680 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cedea-4500-4f8e-9999-f23ad2825f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:19.034303 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cedea-4500-4f8e-9999-f23ad2825f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:19.035899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cedea-4500-4f8e-9999-f23ad2825f00 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:19.045242 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1276/2562] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 10:40:19.049607 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebfc84-a9a3-4c0f-ad73-dd6408d5b6b7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:19.060062 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebfc84-a9a3-4c0f-ad73-dd6408d5b6b7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:19.064761 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:19.065403 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:19.065707 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:19.065979 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] PATH_INFO: `/v3/projects/71b5e8fd10ed4c2a8bdb1abce57809f8/users/fb4180717a2941f28323953296f3177c/roles/94cb07e34dbe4bcabcfaca043a83f461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:19.067924 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b5e8fd10ed4c2a8bdb1abce57809f8, user_id=fb4180717a2941f28323953296f3177c, role_id=94cb07e34dbe4bcabcfaca043a83f461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:19.070589 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:19.081812 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1287/2563] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:19 2022] PUT /identity/v3/projects/71b5e8fd10ed4c2a8bdb1abce57809f8/users/fb4180717a2941f28323953296f3177c/roles/94cb07e34dbe4bcabcfaca043a83f461 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:19.091777 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e0fd-1524-4f33-82f6-b5f521c4a8a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:19.092057 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e0fd-1524-4f33-82f6-b5f521c4a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:19.092325 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e0fd-1524-4f33-82f6-b5f521c4a8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:19.120083 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c9e0e0fd-1524-4f33-82f6-b5f521c4a8a0 None None] MFA Rules not processed for user `fb4180717a2941f28323953296f3177c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:19.121264 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c9e0e0fd-1524-4f33-82f6-b5f521c4a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:19.147351 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:19.189584 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1277/2564] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:19.200177 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:19.201881 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:19.258514 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56cf64-9eec-4413-90e6-1fd12dade22c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:19.262039 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124baf4-a7fb-4e1f-bc10-117a7755ec3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:19.262653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124baf4-a7fb-4e1f-bc10-117a7755ec3f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:19.262932 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124baf4-a7fb-4e1f-bc10-117a7755ec3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:19.263198 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124baf4-a7fb-4e1f-bc10-117a7755ec3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:19.272435 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124baf4-a7fb-4e1f-bc10-117a7755ec3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:19.273852 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124baf4-a7fb-4e1f-bc10-117a7755ec3f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:19.283304 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1288/2565] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:19.649821 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0e0fd-1524-4f33-82f6-b5f521c4a8a0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:19.711783 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0e0fd-1524-4f33-82f6-b5f521c4a8a0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:19.716253 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:19.716870 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:19.717144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:19.717413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] PATH_INFO: `/v3/users/11400830569f4a5a8b2098e9195f6bde` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:19.718549 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11400830569f4a5a8b2098e9195f6bde)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:19.720610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:19.765566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11400830569f4a5a8b2098e9195f6bde'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:19.766203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11400830569f4a5a8b2098e9195f6bde'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:19.773252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11400830569f4a5a8b2098e9195f6bde'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:19.784662 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11400830569f4a5a8b2098e9195f6bde'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:19.785280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11400830569f4a5a8b2098e9195f6bde'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:19.796979 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1278/2566] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:40:19 2022] DELETE /identity/v3/users/11400830569f4a5a8b2098e9195f6bde => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:19.802074 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a37da-e04e-4395-bc37-ca8e80a34f5a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:19.802376 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a37da-e04e-4395-bc37-ca8e80a34f5a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:19.802652 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a37da-e04e-4395-bc37-ca8e80a34f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:19.821586 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-064a37da-e04e-4395-bc37-ca8e80a34f5a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:19.822805 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-064a37da-e04e-4395-bc37-ca8e80a34f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:19.882550 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1289/2567] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:19.892339 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:19.893764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:19.939199 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:19.952427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3959b2f-61bf-4b1c-aac3-582525ff4420 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:19.956065 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833f5b-5282-4223-a253-34755d292b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:19.956656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833f5b-5282-4223-a253-34755d292b00 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:19.956929 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833f5b-5282-4223-a253-34755d292b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:19.957194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833f5b-5282-4223-a253-34755d292b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:19.965973 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15833f5b-5282-4223-a253-34755d292b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:19.967409 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15833f5b-5282-4223-a253-34755d292b00 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:19.977276 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1279/2568] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:20.247718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a37da-e04e-4395-bc37-ca8e80a34f5a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:20.249302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-064a37da-e04e-4395-bc37-ca8e80a34f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:20.263579 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a37da-e04e-4395-bc37-ca8e80a34f5a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:20.269678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:20.270230 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:20.270561 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:20.270828 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] PATH_INFO: `/v3/projects/63a39bd12d894b81b11978c31fd3d3a4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:20.271936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a39bd12d894b81b11978c31fd3d3a4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:20.273920 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:20.305719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:20.321629 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a39bd12d894b81b11978c31fd3d3a4', 'request_id': 'req-09536e07-10eb-4f81-b428-381b7c939da4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.327767 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a39bd12d894b81b11978c31fd3d3a4', 'request_id': 'req-09536e07-10eb-4f81-b428-381b7c939da4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.335659 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a39bd12d894b81b11978c31fd3d3a4', 'request_id': 'req-09536e07-10eb-4f81-b428-381b7c939da4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.352464 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] The token cache is being invalidate because project 63a39bd12d894b81b11978c31fd3d3a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:40:20.352811 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.353474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.355250 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1290/2569] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:40:20 2022] DELETE /identity/v3/projects/63a39bd12d894b81b11978c31fd3d3a4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:20.360993 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833f5b-5282-4223-a253-34755d292b00 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:20.370617 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833f5b-5282-4223-a253-34755d292b00 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:20.426715 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:20.427331 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:20.427610 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:20.427879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] PATH_INFO: `/v3/users/fb4180717a2941f28323953296f3177c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:20.428994 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb4180717a2941f28323953296f3177c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:20.430925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:20.476029 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb4180717a2941f28323953296f3177c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.476671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb4180717a2941f28323953296f3177c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.483819 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb4180717a2941f28323953296f3177c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.495815 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb4180717a2941f28323953296f3177c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.496442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23a51e3c-92d7-4903-b082-52ad371df401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb4180717a2941f28323953296f3177c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.508350 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1280/2570] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:40:20 2022] DELETE /identity/v3/users/fb4180717a2941f28323953296f3177c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:40:20.774182 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:20.775720 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:20.789297 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09536e07-10eb-4f81-b428-381b7c939da4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:20.846526 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:20.847165 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:20.847446 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:20.847718 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] PATH_INFO: `/v3/projects/71b5e8fd10ed4c2a8bdb1abce57809f8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:20.848836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b5e8fd10ed4c2a8bdb1abce57809f8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:20.850818 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:20.897551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b5e8fd10ed4c2a8bdb1abce57809f8', 'request_id': 'req-25ac6132-e286-4ca9-8590-5d43ad29d41c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.903395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b5e8fd10ed4c2a8bdb1abce57809f8', 'request_id': 'req-25ac6132-e286-4ca9-8590-5d43ad29d41c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.910698 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b5e8fd10ed4c2a8bdb1abce57809f8', 'request_id': 'req-25ac6132-e286-4ca9-8590-5d43ad29d41c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.921299 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] The token cache is being invalidate because project 71b5e8fd10ed4c2a8bdb1abce57809f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:40:20.921585 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.922199 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25ac6132-e286-4ca9-8590-5d43ad29d41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:20.923771 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1291/2571] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:40:20 2022] DELETE /identity/v3/projects/71b5e8fd10ed4c2a8bdb1abce57809f8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:20.928772 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ead9e-c2ce-4d1e-a92b-4c9419634722 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:20.929054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ead9e-c2ce-4d1e-a92b-4c9419634722 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:20.929324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ead9e-c2ce-4d1e-a92b-4c9419634722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:20.949697 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c75ead9e-c2ce-4d1e-a92b-4c9419634722 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:20.950953 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c75ead9e-c2ce-4d1e-a92b-4c9419634722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:21.009804 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1281/2572] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:21.017883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8d045-833a-492c-901a-dc466554c2bf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.018166 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8d045-833a-492c-901a-dc466554c2bf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.018474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8d045-833a-492c-901a-dc466554c2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.037703 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ddd8d045-833a-492c-901a-dc466554c2bf None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:21.038909 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ddd8d045-833a-492c-901a-dc466554c2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:21.046678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1292/2573] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:21.051531 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ead9e-c2ce-4d1e-a92b-4c9419634722 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:21.057442 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:21.061220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ead9e-c2ce-4d1e-a92b-4c9419634722 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:21.068121 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:21.068867 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.069220 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.069492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.070145 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:40:21.070504 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:21.072095 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:21.084227 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1282/2574] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:40:21.088556 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8d045-833a-492c-901a-dc466554c2bf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:21.097839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8d045-833a-492c-901a-dc466554c2bf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:21.102678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebaee3-285d-4246-96cb-c5d6a07e9a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:21.103269 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebaee3-285d-4246-96cb-c5d6a07e9a21 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.103541 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebaee3-285d-4246-96cb-c5d6a07e9a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.103804 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebaee3-285d-4246-96cb-c5d6a07e9a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.104432 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebaee3-285d-4246-96cb-c5d6a07e9a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:21.106377 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebaee3-285d-4246-96cb-c5d6a07e9a21 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:21.134407 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1293/2575] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:40:21 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:21.138751 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:21.148921 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e84ae-7541-4447-ae4e-17277bd88a66 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:21.206016 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506272f-24e4-4b85-b2c8-50c5b8904920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:21.206633 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506272f-24e4-4b85-b2c8-50c5b8904920 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.206904 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506272f-24e4-4b85-b2c8-50c5b8904920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.207169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506272f-24e4-4b85-b2c8-50c5b8904920 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.207790 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506272f-24e4-4b85-b2c8-50c5b8904920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:21.209667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506272f-24e4-4b85-b2c8-50c5b8904920 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:21.230547 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1283/2576] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:40:21 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:21.234875 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebaee3-285d-4246-96cb-c5d6a07e9a21 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:21.246102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebaee3-285d-4246-96cb-c5d6a07e9a21 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:21.250623 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ebe64-9646-4917-af20-2f4c4927672f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:21.252320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ebe64-9646-4917-af20-2f4c4927672f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.252596 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ebe64-9646-4917-af20-2f4c4927672f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.252863 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ebe64-9646-4917-af20-2f4c4927672f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.253492 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ebe64-9646-4917-af20-2f4c4927672f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:21.255102 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ebe64-9646-4917-af20-2f4c4927672f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:21.264841 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1294/2577] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:21.268791 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506272f-24e4-4b85-b2c8-50c5b8904920 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:21.278574 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506272f-24e4-4b85-b2c8-50c5b8904920 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:21.283310 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bca3d-214d-4fb0-9a6b-3b627b856f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:21.283888 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bca3d-214d-4fb0-9a6b-3b627b856f9b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.284162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bca3d-214d-4fb0-9a6b-3b627b856f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.284428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bca3d-214d-4fb0-9a6b-3b627b856f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.285060 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bca3d-214d-4fb0-9a6b-3b627b856f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:21.286667 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bca3d-214d-4fb0-9a6b-3b627b856f9b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:21.297185 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1284/2578] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:21.301513 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ebe64-9646-4917-af20-2f4c4927672f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:21.311208 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ebe64-9646-4917-af20-2f4c4927672f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:21.316488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:21.317127 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.317406 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.317678 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] PATH_INFO: `/v3/projects/d408232b05f3448d8192ca3505f4c3a1/users/2b0a803e33bb46f8b04a888a41a14cfb/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.319883 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:21.334876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d408232b05f3448d8192ca3505f4c3a1, user_id=2b0a803e33bb46f8b04a888a41a14cfb, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:21.337562 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:21.350612 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1295/2579] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:21 2022] PUT /identity/v3/projects/d408232b05f3448d8192ca3505f4c3a1/users/2b0a803e33bb46f8b04a888a41a14cfb/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:40:21.361834 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0607-951a-4b7f-945a-45f72f872cdc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.362377 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0607-951a-4b7f-945a-45f72f872cdc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.362486 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0607-951a-4b7f-945a-45f72f872cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.389671 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4c5c0607-951a-4b7f-945a-45f72f872cdc None None] MFA Rules not processed for user `2b0a803e33bb46f8b04a888a41a14cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:21.390906 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4c5c0607-951a-4b7f-945a-45f72f872cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:21.453258 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1285/2580] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:21.464690 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:21.466454 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:21.523488 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7148a2a-20d0-4d30-a56f-a7f442977309 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:21.527174 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83339030-2798-4586-b0d6-6e90f8c190d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:21.527791 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83339030-2798-4586-b0d6-6e90f8c190d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:21.528088 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83339030-2798-4586-b0d6-6e90f8c190d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:21.528353 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83339030-2798-4586-b0d6-6e90f8c190d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:21.537342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83339030-2798-4586-b0d6-6e90f8c190d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:21.538792 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83339030-2798-4586-b0d6-6e90f8c190d1 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:21.547345 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1296/2581] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:29.933324 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0607-951a-4b7f-945a-45f72f872cdc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:29.995345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0607-951a-4b7f-945a-45f72f872cdc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:29.999800 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:30.000418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.000691 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.000959 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] PATH_INFO: `/v3/users/2b0a803e33bb46f8b04a888a41a14cfb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.002104 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b0a803e33bb46f8b04a888a41a14cfb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:30.004169 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:30.005598 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:30.053435 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b0a803e33bb46f8b04a888a41a14cfb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.054072 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b0a803e33bb46f8b04a888a41a14cfb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.061216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b0a803e33bb46f8b04a888a41a14cfb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.073676 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b0a803e33bb46f8b04a888a41a14cfb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.074428 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b0a803e33bb46f8b04a888a41a14cfb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.088091 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1286/2582] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:40:29 2022] DELETE /identity/v3/users/2b0a803e33bb46f8b04a888a41a14cfb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:30.092637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68596e0-a8bb-4ffa-884d-6b80896e079f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.093118 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68596e0-a8bb-4ffa-884d-6b80896e079f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.093278 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68596e0-a8bb-4ffa-884d-6b80896e079f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.113395 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b68596e0-a8bb-4ffa-884d-6b80896e079f None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:30.114499 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b68596e0-a8bb-4ffa-884d-6b80896e079f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:30.173263 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1297/2583] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:30.183089 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:30.184522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:30.241208 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a90466-98a9-401d-8ca8-ecb183a2b146 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:30.245049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715149b4-db90-46ba-bfe6-aebd25bb994a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:30.245644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715149b4-db90-46ba-bfe6-aebd25bb994a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.245916 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715149b4-db90-46ba-bfe6-aebd25bb994a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.246234 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715149b4-db90-46ba-bfe6-aebd25bb994a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.255358 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715149b4-db90-46ba-bfe6-aebd25bb994a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:30.256764 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715149b4-db90-46ba-bfe6-aebd25bb994a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:30.266226 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1287/2584] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:30.537059 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68596e0-a8bb-4ffa-884d-6b80896e079f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:30.538534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b68596e0-a8bb-4ffa-884d-6b80896e079f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:30.550970 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68596e0-a8bb-4ffa-884d-6b80896e079f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:30.555486 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:30.556111 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.556383 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.556644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] PATH_INFO: `/v3/projects/d408232b05f3448d8192ca3505f4c3a1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.557743 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d408232b05f3448d8192ca3505f4c3a1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:30.559684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:30.607653 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd408232b05f3448d8192ca3505f4c3a1', 'request_id': 'req-53e4f15d-56d1-447b-a75f-b7d142b02080'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.614081 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd408232b05f3448d8192ca3505f4c3a1', 'request_id': 'req-53e4f15d-56d1-447b-a75f-b7d142b02080'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.621389 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd408232b05f3448d8192ca3505f4c3a1', 'request_id': 'req-53e4f15d-56d1-447b-a75f-b7d142b02080'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.632319 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] The token cache is being invalidate because project d408232b05f3448d8192ca3505f4c3a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:40:30.632603 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.633223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-53e4f15d-56d1-447b-a75f-b7d142b02080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:40:30.634739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1298/2585] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:40:30 2022] DELETE /identity/v3/projects/d408232b05f3448d8192ca3505f4c3a1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:30.639652 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddeb06-9b6f-4318-a095-1f2810a18938 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.639930 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddeb06-9b6f-4318-a095-1f2810a18938 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.640200 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddeb06-9b6f-4318-a095-1f2810a18938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.660903 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-86ddeb06-9b6f-4318-a095-1f2810a18938 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:30.662070 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86ddeb06-9b6f-4318-a095-1f2810a18938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:30.720657 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1288/2586] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:30.729565 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411e5a-51a1-4cbf-818a-a745f188044b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.729848 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411e5a-51a1-4cbf-818a-a745f188044b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.730122 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411e5a-51a1-4cbf-818a-a745f188044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.754994 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5c411e5a-51a1-4cbf-818a-a745f188044b None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:30.756232 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5c411e5a-51a1-4cbf-818a-a745f188044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:30.764043 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1299/2587] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:30.768492 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddeb06-9b6f-4318-a095-1f2810a18938 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:30.778141 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddeb06-9b6f-4318-a095-1f2810a18938 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:30.782648 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:30.783305 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.783824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.783824 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.784499 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:40:30.784799 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:30.786430 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:30.797925 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1289/2588] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:40:30.802455 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c411e5a-51a1-4cbf-818a-a745f188044b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:30.806751 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:30.812991 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c411e5a-51a1-4cbf-818a-a745f188044b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:30.819577 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8508cf5-e6a4-4df9-b7dc-674a43bfe822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:30.820171 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8508cf5-e6a4-4df9-b7dc-674a43bfe822 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.820443 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8508cf5-e6a4-4df9-b7dc-674a43bfe822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.820711 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8508cf5-e6a4-4df9-b7dc-674a43bfe822 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.821339 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8508cf5-e6a4-4df9-b7dc-674a43bfe822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:30.823262 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8508cf5-e6a4-4df9-b7dc-674a43bfe822 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:30.851508 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1300/2589] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:40:30 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:30.855467 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:30.866477 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad18cdd-2ce2-4c71-812d-d5b2c6c2e864 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:30.870955 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:30.922784 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ca962-6726-4ebd-87d9-5e9ddb5682ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:30.923374 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca962-6726-4ebd-87d9-5e9ddb5682ed admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.923655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca962-6726-4ebd-87d9-5e9ddb5682ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.924494 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca962-6726-4ebd-87d9-5e9ddb5682ed admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.925415 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ca962-6726-4ebd-87d9-5e9ddb5682ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:30.927343 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ca962-6726-4ebd-87d9-5e9ddb5682ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:30.948644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1290/2590] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:40:30 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:40:30.952922 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8508cf5-e6a4-4df9-b7dc-674a43bfe822 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:30.963009 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8508cf5-e6a4-4df9-b7dc-674a43bfe822 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:30.967551 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b1515-de84-430a-9890-db7282379bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:30.968141 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b1515-de84-430a-9890-db7282379bef admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:30.968417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b1515-de84-430a-9890-db7282379bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:30.968684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b1515-de84-430a-9890-db7282379bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:30.969320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b1515-de84-430a-9890-db7282379bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:30.971009 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b1515-de84-430a-9890-db7282379bef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:30.980574 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1301/2591] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:30.984801 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ca962-6726-4ebd-87d9-5e9ddb5682ed admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:30.995162 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ca962-6726-4ebd-87d9-5e9ddb5682ed admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:30.999692 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27225f76-5635-4a14-9dee-7c3ad21df651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:31.000261 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27225f76-5635-4a14-9dee-7c3ad21df651 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:31.000532 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27225f76-5635-4a14-9dee-7c3ad21df651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:31.000796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27225f76-5635-4a14-9dee-7c3ad21df651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:31.001422 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27225f76-5635-4a14-9dee-7c3ad21df651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:31.003645 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27225f76-5635-4a14-9dee-7c3ad21df651 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:31.014046 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1291/2592] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:40:31.018194 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b1515-de84-430a-9890-db7282379bef admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:31.027915 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b1515-de84-430a-9890-db7282379bef admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:31.032523 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:31.033159 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:31.033463 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:31.033752 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] PATH_INFO: `/v3/projects/ec15fb49425349a39e68aa7b875a3ff5/users/6b978a50c8144f0d84aa2e83ae88b56f/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:31.047846 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec15fb49425349a39e68aa7b875a3ff5, user_id=6b978a50c8144f0d84aa2e83ae88b56f, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:31.051310 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:31.063739 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1302/2593] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:40:31 2022] PUT /identity/v3/projects/ec15fb49425349a39e68aa7b875a3ff5/users/6b978a50c8144f0d84aa2e83ae88b56f/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:40:31.074911 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9319-1b52-4387-a1be-73c64f615a2e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:31.075203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9319-1b52-4387-a1be-73c64f615a2e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:31.075459 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9319-1b52-4387-a1be-73c64f615a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:31.103101 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4fee9319-1b52-4387-a1be-73c64f615a2e None None] MFA Rules not processed for user `6b978a50c8144f0d84aa2e83ae88b56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:40:31.104280 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4fee9319-1b52-4387-a1be-73c64f615a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:31.166537 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1292/2594] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:40:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:40:31.179839 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:40:31.181500 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:40:31.257672 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab8045-26cd-4845-973f-f2e47331c4d8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:40:31.261315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5af4e9-d434-457f-92bb-a4677e0dbb17 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ac934ed0122470cb489d6b324c664f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:40:31.261907 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5af4e9-d434-457f-92bb-a4677e0dbb17 service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:40:31.262181 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5af4e9-d434-457f-92bb-a4677e0dbb17 service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:40:31.262483 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5af4e9-d434-457f-92bb-a4677e0dbb17 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:40:31.270618 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5af4e9-d434-457f-92bb-a4677e0dbb17 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:40:31.272031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5af4e9-d434-457f-92bb-a4677e0dbb17 service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:40:31.280592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1303/2595] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:40:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:40:41.163567 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:40:41.276675 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=58,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:41:10.969853 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9319-1b52-4387-a1be-73c64f615a2e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:11.031365 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9319-1b52-4387-a1be-73c64f615a2e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:11.035796 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:11.036403 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.036884 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.037161 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] PATH_INFO: `/v3/users/6b978a50c8144f0d84aa2e83ae88b56f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.038271 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b978a50c8144f0d84aa2e83ae88b56f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:11.040180 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:11.091032 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b978a50c8144f0d84aa2e83ae88b56f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.091682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b978a50c8144f0d84aa2e83ae88b56f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.098569 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b978a50c8144f0d84aa2e83ae88b56f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.111014 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b978a50c8144f0d84aa2e83ae88b56f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.111630 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b978a50c8144f0d84aa2e83ae88b56f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.125283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1293/2596] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:41:10 2022] DELETE /identity/v3/users/6b978a50c8144f0d84aa2e83ae88b56f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:41:11.129765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271da56e-5525-4a08-a462-ccc50c94c1d4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.130252 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271da56e-5525-4a08-a462-ccc50c94c1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.130417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271da56e-5525-4a08-a462-ccc50c94c1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.149940 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-271da56e-5525-4a08-a462-ccc50c94c1d4 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:41:11.151149 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-271da56e-5525-4a08-a462-ccc50c94c1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:11.210222 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1304/2597] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:41:11.219857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:11.221222 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:11.281119 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aeed3-01e1-430c-a15e-1934d317d9c5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:11.285194 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e6583-a6b6-457b-9126-8d361b57ce17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:11.285832 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6583-a6b6-457b-9126-8d361b57ce17 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.286216 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6583-a6b6-457b-9126-8d361b57ce17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.287249 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6583-a6b6-457b-9126-8d361b57ce17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.296752 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e6583-a6b6-457b-9126-8d361b57ce17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:11.298172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e6583-a6b6-457b-9126-8d361b57ce17 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:11.308321 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1294/2598] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:41:11.575878 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271da56e-5525-4a08-a462-ccc50c94c1d4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:11.577447 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-271da56e-5525-4a08-a462-ccc50c94c1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:11.590006 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271da56e-5525-4a08-a462-ccc50c94c1d4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:11.595968 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:11.596597 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.596986 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.597126 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] PATH_INFO: `/v3/projects/ec15fb49425349a39e68aa7b875a3ff5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.598234 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec15fb49425349a39e68aa7b875a3ff5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:11.601599 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:11.650631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec15fb49425349a39e68aa7b875a3ff5', 'request_id': 'req-0453aed3-7b6d-4215-8102-4319f629ffa7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.656732 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec15fb49425349a39e68aa7b875a3ff5', 'request_id': 'req-0453aed3-7b6d-4215-8102-4319f629ffa7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.663910 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec15fb49425349a39e68aa7b875a3ff5', 'request_id': 'req-0453aed3-7b6d-4215-8102-4319f629ffa7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.674105 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] The token cache is being invalidate because project ec15fb49425349a39e68aa7b875a3ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:41:11.674417 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.675031 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0453aed3-7b6d-4215-8102-4319f629ffa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:41:11.676507 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1305/2599] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:41:11 2022] DELETE /identity/v3/projects/ec15fb49425349a39e68aa7b875a3ff5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:41:11.681935 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601f113-862c-4304-8f1f-45be7ed34d78 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.682219 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601f113-862c-4304-8f1f-45be7ed34d78 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.682522 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601f113-862c-4304-8f1f-45be7ed34d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.702378 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6601f113-862c-4304-8f1f-45be7ed34d78 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:41:11.703540 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6601f113-862c-4304-8f1f-45be7ed34d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:11.719826 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:41:11.764855 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1295/2600] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:41:11.774622 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2eebe-214d-482d-8ebc-078440eda489 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.774906 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2eebe-214d-482d-8ebc-078440eda489 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.775179 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2eebe-214d-482d-8ebc-078440eda489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.795160 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-08f2eebe-214d-482d-8ebc-078440eda489 None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:41:11.796342 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-08f2eebe-214d-482d-8ebc-078440eda489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:11.804033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1306/2601] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:41:11.808166 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601f113-862c-4304-8f1f-45be7ed34d78 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:11.817558 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601f113-862c-4304-8f1f-45be7ed34d78 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:11.822056 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:11.822656 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.822928 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.823188 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.823822 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 10:41:11.824110 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:11.825655 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:11.837427 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1296/2602] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Oct 17 10:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 10:41:11.841571 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2eebe-214d-482d-8ebc-078440eda489 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:11.851976 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2eebe-214d-482d-8ebc-078440eda489 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:11.856497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78487-edb9-4ed2-98c6-0ac61bac5105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:11.857076 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78487-edb9-4ed2-98c6-0ac61bac5105 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.857379 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78487-edb9-4ed2-98c6-0ac61bac5105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.857644 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78487-edb9-4ed2-98c6-0ac61bac5105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.858533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b78487-edb9-4ed2-98c6-0ac61bac5105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:11.860411 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b78487-edb9-4ed2-98c6-0ac61bac5105 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:11.889471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1307/2603] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Oct 17 10:41:11 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:41:11.893615 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:11.903836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c6031-5757-4188-95bf-cf6d84039530 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:11.960019 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa7bc6-6d14-42a4-9e31-897711fa90c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:11.960606 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa7bc6-6d14-42a4-9e31-897711fa90c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:11.960879 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa7bc6-6d14-42a4-9e31-897711fa90c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:11.961144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa7bc6-6d14-42a4-9e31-897711fa90c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:11.961771 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa7bc6-6d14-42a4-9e31-897711fa90c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:11.963688 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa7bc6-6d14-42a4-9e31-897711fa90c6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:11.984387 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1297/2604] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Oct 17 10:41:11 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 10:41:11.988637 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78487-edb9-4ed2-98c6-0ac61bac5105 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:11.999324 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78487-edb9-4ed2-98c6-0ac61bac5105 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:12.003849 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9cd17-89b9-4927-b65e-a4ce6c7d55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:12.004429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9cd17-89b9-4927-b65e-a4ce6c7d55a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:12.004705 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9cd17-89b9-4927-b65e-a4ce6c7d55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:12.004978 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9cd17-89b9-4927-b65e-a4ce6c7d55a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:12.006108 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9cd17-89b9-4927-b65e-a4ce6c7d55a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:12.007719 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9cd17-89b9-4927-b65e-a4ce6c7d55a4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:12.017594 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1308/2605] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:41:12.022172 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa7bc6-6d14-42a4-9e31-897711fa90c6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:12.032054 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa7bc6-6d14-42a4-9e31-897711fa90c6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:12.036663 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeccb6b-4cec-4d62-930b-5d222f97d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:12.037232 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeccb6b-4cec-4d62-930b-5d222f97d1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:12.037507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeccb6b-4cec-4d62-930b-5d222f97d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:12.037762 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeccb6b-4cec-4d62-930b-5d222f97d1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:12.038432 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeccb6b-4cec-4d62-930b-5d222f97d1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:12.040030 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeccb6b-4cec-4d62-930b-5d222f97d1ca admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:12.050304 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1298/2606] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Oct 17 10:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 10:41:12.054648 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9cd17-89b9-4927-b65e-a4ce6c7d55a4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:12.064424 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9cd17-89b9-4927-b65e-a4ce6c7d55a4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:12.069747 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:12.070468 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:12.070775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:12.071063 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] PATH_INFO: `/v3/projects/f4a5813ff7244904ad8d2ac132c63cf9/users/409d3f7190e6423193765243268c7ceb/roles/cbe9824e14834e26a81ef6a408bd1bd4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:12.087461 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a5813ff7244904ad8d2ac132c63cf9, user_id=409d3f7190e6423193765243268c7ceb, role_id=cbe9824e14834e26a81ef6a408bd1bd4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:12.090156 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:12.103370 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1309/2607] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Oct 17 10:41:12 2022] PUT /identity/v3/projects/f4a5813ff7244904ad8d2ac132c63cf9/users/409d3f7190e6423193765243268c7ceb/roles/cbe9824e14834e26a81ef6a408bd1bd4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:41:12.108635 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0f689-dde3-4cc6-a96b-a471e03e473a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:12.108908 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0f689-dde3-4cc6-a96b-a471e03e473a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:12.109496 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0f689-dde3-4cc6-a96b-a471e03e473a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:12.128753 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b9e0f689-dde3-4cc6-a96b-a471e03e473a None None] MFA Rules not processed for user `e9716bb37d07416391c7fe0f922727e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:41:12.129910 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b9e0f689-dde3-4cc6-a96b-a471e03e473a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:12.188049 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1299/2608] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:41:12.198757 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:12.200474 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:12.222944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:41:12.260228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16033a9f-7fa2-4931-b454-e4c063d31615 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:12.263880 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:12.264471 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:12.264745 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:12.265026 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:12.273902 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:12.275378 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:12.284516 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1310/2609] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:41:12.904947 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4cb9-7487-4ff7-92cd-90484280dcd7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:12.905329 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4cb9-7487-4ff7-92cd-90484280dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:12.905669 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4cb9-7487-4ff7-92cd-90484280dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:12.934696 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ef2f4cb9-7487-4ff7-92cd-90484280dcd7 None None] MFA Rules not processed for user `409d3f7190e6423193765243268c7ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:41:12.935897 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef2f4cb9-7487-4ff7-92cd-90484280dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:12.952670 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:41:13.005159 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1300/2610] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:41:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:41:13.015873 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:13.017241 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:13.080068 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce1bbe-bea2-4a7b-bcde-bf013d0aae7d service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:13.084187 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf4e1a-5370-4568-9756-707589302648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f1a6c05ed94f5fa81dd355497b6754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:13.084876 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4e1a-5370-4568-9756-707589302648 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:13.085064 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4e1a-5370-4568-9756-707589302648 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:13.085315 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4e1a-5370-4568-9756-707589302648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:13.094176 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf4e1a-5370-4568-9756-707589302648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:13.095640 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf4e1a-5370-4568-9756-707589302648 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:13.104822 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1311/2611] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:41:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:41:14.464813 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2786d-3002-4c1d-98c8-78e161a84b74 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:14.465355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2786d-3002-4c1d-98c8-78e161a84b74 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:14.465355 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2786d-3002-4c1d-98c8-78e161a84b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:14.485270 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-74e2786d-3002-4c1d-98c8-78e161a84b74 None None] MFA Rules not processed for user `409d3f7190e6423193765243268c7ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:41:14.486506 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-74e2786d-3002-4c1d-98c8-78e161a84b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:14.493543 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1301/2612] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:41:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:41:14.724808 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf4e1a-5370-4568-9756-707589302648 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:14.734534 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf4e1a-5370-4568-9756-707589302648 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:14.738008 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:14.738631 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:14.738905 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:14.739172 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:14.741302 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:14.750836 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:14.752245 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:14.760759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1312/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:41:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:41:16.086244 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2786d-3002-4c1d-98c8-78e161a84b74 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:16.087644 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-74e2786d-3002-4c1d-98c8-78e161a84b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:16.157433 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2786d-3002-4c1d-98c8-78e161a84b74 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:16.163000 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:16.163596 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:16.163864 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:16.164138 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:16.165899 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:16.177103 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:16.178836 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:16.188710 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1302/2614] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:41:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:41:17.969807 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:17.979228 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536a696-6600-4579-9958-aea94e62370b service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:17.982842 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81294c06-e714-4b05-b0bc-cf29a6b31dc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7883e3f6ab4f58969f51ffd439b1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:17.983684 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81294c06-e714-4b05-b0bc-cf29a6b31dc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:17.983959 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81294c06-e714-4b05-b0bc-cf29a6b31dc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:17.984223 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81294c06-e714-4b05-b0bc-cf29a6b31dc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:17.985936 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-81294c06-e714-4b05-b0bc-cf29a6b31dc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:17.995521 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81294c06-e714-4b05-b0bc-cf29a6b31dc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:17.996989 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81294c06-e714-4b05-b0bc-cf29a6b31dc2 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:18.004974 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1313/2615] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:41:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:41:18.638413 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:41:18.640594 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:18.725604 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428b4c-1314-4c38-a6ec-19378ddd1819 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:41:18.731077 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:41:18.731729 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:41:18.732005 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:41:18.732275 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:41:18.734237 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:41:18.747055 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:41:18.748612 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:41:18.759346 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1303/2616] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:41:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:41:28.001271 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=55 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:41:28.754678 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:42:29.248444 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dfcb-9544-4e9c-b8d7-a54d4e84bd31 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:42:29.248973 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dfcb-9544-4e9c-b8d7-a54d4e84bd31 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:42:29.249083 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dfcb-9544-4e9c-b8d7-a54d4e84bd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:42:29.268033 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ec38dfcb-9544-4e9c-b8d7-a54d4e84bd31 None None] MFA Rules not processed for user `409d3f7190e6423193765243268c7ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 10:42:29.269211 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ec38dfcb-9544-4e9c-b8d7-a54d4e84bd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:42:29.276312 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1314/2617] 10.222.0.16 () {58 vars in 958 bytes} [Mon Oct 17 10:42:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 10:42:37.583763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:42:37.585306 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:42:37.601763 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e764cd4-8290-4281-823c-9cbc1f460da4 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:42:37.607283 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d52624718de4457adc44e9965cd7a09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:42:37.607876 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:42:37.608149 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:42:37.608417 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:42:37.610399 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:42:37.623122 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:42:37.624787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:42:37.635478 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1304/2618] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:42:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:42:39.263799 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:42:47.630958 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:43:31.688602 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38dfcb-9544-4e9c-b8d7-a54d4e84bd31 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:43:31.719592 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38dfcb-9544-4e9c-b8d7-a54d4e84bd31 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:43:31.727944 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:43:31.729137 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:43:31.729654 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:43:31.730144 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:43:31.733729 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:43:31.753566 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:43:31.763459 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:43:31.765272 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:43:31.776642 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1315/2619] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:43:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:43:31.784338 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c8bd-2daa-4a7a-b9ec-981556c621fc service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:43:31.790857 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:43:31.791511 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:43:31.791787 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:43:31.792051 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] PATH_INFO: `/v3/users/409d3f7190e6423193765243268c7ceb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:43:31.793575 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=409d3f7190e6423193765243268c7ceb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:43:31.795097 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:43:31.844507 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409d3f7190e6423193765243268c7ceb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:31.845144 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409d3f7190e6423193765243268c7ceb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:31.853583 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409d3f7190e6423193765243268c7ceb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:31.866294 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409d3f7190e6423193765243268c7ceb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:31.866960 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409d3f7190e6423193765243268c7ceb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:31.878831 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1305/2620] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Oct 17 10:43:31 2022] DELETE /identity/v3/users/409d3f7190e6423193765243268c7ceb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 10:43:32.055801 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:43:32.066611 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f261ae-8994-4282-83cf-48da799d65ad service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:43:32.129000 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9716bb37d07416391c7fe0f922727e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6150f4298a94f959838ad2351eaefb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:43:32.129682 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:43:32.129980 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:43:32.130248 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] PATH_INFO: `/v3/projects/f4a5813ff7244904ad8d2ac132c63cf9` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:43:32.131381 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a5813ff7244904ad8d2ac132c63cf9)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:43:32.133320 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:43:32.180536 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a5813ff7244904ad8d2ac132c63cf9', 'request_id': 'req-9589a02e-3174-43fe-9c82-abc3eb9a3fba'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:32.186759 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a5813ff7244904ad8d2ac132c63cf9', 'request_id': 'req-9589a02e-3174-43fe-9c82-abc3eb9a3fba'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:32.194429 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a5813ff7244904ad8d2ac132c63cf9', 'request_id': 'req-9589a02e-3174-43fe-9c82-abc3eb9a3fba'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:32.204775 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] The token cache is being invalidate because project f4a5813ff7244904ad8d2ac132c63cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 10:43:32.205069 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:32.205688 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 10:43:32.207314 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1316/2621] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Oct 17 10:43:32 2022] DELETE /identity/v3/projects/f4a5813ff7244904ad8d2ac132c63cf9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 10:43:41.875345 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:43:42.201734 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:43:59.953943 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:43:59.955470 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:44:00.070562 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbe9de-5049-49f2-bd13-eada8a6ddf21 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:44:00.129682 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ef807-ec86-4152-9f4a-76e3bf8c8456 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:44:00.130203 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ef807-ec86-4152-9f4a-76e3bf8c8456 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:44:00.131127 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ef807-ec86-4152-9f4a-76e3bf8c8456 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:44:00.131418 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ef807-ec86-4152-9f4a-76e3bf8c8456 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:44:00.133133 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-518ef807-ec86-4152-9f4a-76e3bf8c8456 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:44:00.144080 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ef807-ec86-4152-9f4a-76e3bf8c8456 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:44:00.145632 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ef807-ec86-4152-9f4a-76e3bf8c8456 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:44:00.156410 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1306/2622] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:43:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 10:44:10.151252 dv-cinder-860209-5-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=79 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 10:49:06.800758 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 10:49:06.802158 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 10:49:06.827533 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589a02e-3174-43fe-9c82-abc3eb9a3fba admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 10:49:06.833558 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3fecf-5d1f-480f-bce8-16433f6729db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb92e22612b400dbf390e149abbb105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb915634f9884b49b79a10379df4b0d3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 10:49:06.834201 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3fecf-5d1f-480f-bce8-16433f6729db service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 10:49:06.834497 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3fecf-5d1f-480f-bce8-16433f6729db service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 10:49:06.834765 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3fecf-5d1f-480f-bce8-16433f6729db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 10:49:06.843331 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3fecf-5d1f-480f-bce8-16433f6729db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 10:49:06.844913 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3fecf-5d1f-480f-bce8-16433f6729db service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 10:49:06.854888 dv-cinder-860209-5-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1317/2623] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Oct 17 10:49:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)